[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 17:59:58 dist-test-slave-vr57 krb5kdc[15925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 17:59:58 dist-test-slave-vr57 krb5kdc[15925](info): set up 2 sockets
Sep 23 17:59:58 dist-test-slave-vr57 krb5kdc[15925](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260923 17:59:59.948060 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:00 dist-test-slave-vr57 krb5kdc[15925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:00:00.079555 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36997
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:36997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:00.193291 15938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:00.193645 15938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:00.193730 15938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:00.197829 15938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:00.197938 15938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:00.197984 15938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:00.198050 15938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:00.198093 15938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:00.203490 15938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:36997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:00.204685 15938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:00.206182 15938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:00.216054 15944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:00.215987 15946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:00.216020 15943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:00.216198 15938 server_base.cc:1034] running on GCE node
I20260923 18:00:00.216874 15938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:00.218016 15938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:00.219225 15938 hybrid_clock.cc:648] HybridClock initialized: now 1790186400219203 us; error 36 us; skew 500 ppm
Sep 23 18:00:00 dist-test-slave-vr57 krb5kdc[15925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186400, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:00.222266 15938 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:00:00.223225 15938 webserver.cc:460] Webserver started at http://127.15.139.190:44883/ using document root <none> and password file <none>
I20260923 18:00:00.223850 15938 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:00.223918 15938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:00.224094 15938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:00.226214 15938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "d54717fa5faa44d99a3f960252f88fde"
format_stamp: "Formatted at 2026-09-23 18:00:00 on dist-test-slave-vr57"
server_key: "0458fe249e915bd85056dc9eb13f2aeb"
server_key_iv: "adede933fb1b0d8800c6cfab863de5f3"
server_key_version: "encryptionkey@0"
I20260923 18:00:00.226796 15938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "d54717fa5faa44d99a3f960252f88fde"
format_stamp: "Formatted at 2026-09-23 18:00:00 on dist-test-slave-vr57"
server_key: "0458fe249e915bd85056dc9eb13f2aeb"
server_key_iv: "adede933fb1b0d8800c6cfab863de5f3"
server_key_version: "encryptionkey@0"
I20260923 18:00:00.230784 15938 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:00:00.233373 15953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:00.234766 15938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:00.234902 15938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "d54717fa5faa44d99a3f960252f88fde"
format_stamp: "Formatted at 2026-09-23 18:00:00 on dist-test-slave-vr57"
server_key: "0458fe249e915bd85056dc9eb13f2aeb"
server_key_iv: "adede933fb1b0d8800c6cfab863de5f3"
server_key_version: "encryptionkey@0"
I20260923 18:00:00.235015 15938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:00.250799 15938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:00.254364 15938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:00.254592 15938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:00.262480 15938 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:36997
I20260923 18:00:00.262514 16005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:36997 every 8 connection(s)
I20260923 18:00:00.263665 15938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:00.266463 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 15938
I20260923 18:00:00.266633 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:00.266769 16006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:00.268329 15918 external_mini_cluster.cc:1428] Setting key 2e72d40eb4bb71f27a7cf6b49b1500c1
I20260923 18:00:00.273355 16006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde: Bootstrap starting.
I20260923 18:00:00.275585 16006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:00.276310 16006 log.cc:824] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:00.278250 16006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde: No bootstrap required, opened a new log
I20260923 18:00:00.281703 16006 raft_consensus.cc:348] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d54717fa5faa44d99a3f960252f88fde" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36997 } }
I20260923 18:00:00.282001 16006 raft_consensus.cc:374] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:00.282078 16006 raft_consensus.cc:729] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d54717fa5faa44d99a3f960252f88fde, State: Initialized, Role: FOLLOWER
I20260923 18:00:00.282567 16006 consensus_queue.cc:260] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d54717fa5faa44d99a3f960252f88fde" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36997 } }
I20260923 18:00:00.282721 16006 raft_consensus.cc:388] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:00.282806 16006 raft_consensus.cc:482] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:00.282943 16006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:00.283911 16006 raft_consensus.cc:504] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d54717fa5faa44d99a3f960252f88fde" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36997 } }
I20260923 18:00:00.284315 16006 leader_election.cc:302] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d54717fa5faa44d99a3f960252f88fde; no voters:
I20260923 18:00:00.284688 16006 leader_election.cc:288] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:00.284750 16011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:00.284956 16011 raft_consensus.cc:686] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [term 1 LEADER]: Becoming Leader. State: Replica: d54717fa5faa44d99a3f960252f88fde, State: Running, Role: LEADER
I20260923 18:00:00.285308 16011 consensus_queue.cc:237] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d54717fa5faa44d99a3f960252f88fde" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36997 } }
I20260923 18:00:00.285966 16006 sys_catalog.cc:564] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [sys.catalog]: configured and running, proceeding with master startup.
Sep 23 18:00:00 dist-test-slave-vr57 krb5kdc[15925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:00.288800 16012 sys_catalog.cc:455] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d54717fa5faa44d99a3f960252f88fde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d54717fa5faa44d99a3f960252f88fde" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36997 } } }
I20260923 18:00:00.289217 16012 sys_catalog.cc:458] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:00.289619 16013 sys_catalog.cc:455] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [sys.catalog]: SysCatalogTable state changed. Reason: New leader d54717fa5faa44d99a3f960252f88fde. Latest consensus state: current_term: 1 leader_uuid: "d54717fa5faa44d99a3f960252f88fde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d54717fa5faa44d99a3f960252f88fde" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36997 } } }
I20260923 18:00:00.289728 16013 sys_catalog.cc:458] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde [sys.catalog]: This master's current role is: LEADER
W20260923 18:00:00.291832 16026 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:00:00.291941 16026 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:00:00.292012 16027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:00.292860 16027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:00.293591 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:00.272038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35774 (local address 127.15.139.190:36997)
0923 18:00:00.272540 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:00.272560 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:00.273571 (+ 1011us) server_negotiation.cc:407] Connection header received
0923 18:00:00.276872 (+ 3301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:00.276889 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:00.277182 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:00.277521 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:00.279823 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:00.280836 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:00.282220 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:00.282667 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:00.282844 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:00.287749 (+ 4905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:00.287755 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:00.287771 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:00.287807 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:00.290091 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:00.290674 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:00.290687 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:00.290692 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:00.290805 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:00.291173 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:00.291176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:00.291178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:00.291667 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:00.291874 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:00.292478 (+ 604us) server_negotiation.cc:299] Negotiation successful
0923 18:00:00.292745 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":150,"threads_started":1}
I20260923 18:00:00.298520 16027 catalog_manager.cc:1347] Generated new cluster ID: db3d9b02eb0e414cae013c914fe92ae3
I20260923 18:00:00.298597 16027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:00.334856 16027 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:00.335952 16027 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:00.356217 16027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d54717fa5faa44d99a3f960252f88fde: Generated new TSK 0
I20260923 18:00:00.357124 16027 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:00:00.399225 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36997
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:00.513156 16032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:00.513412 16032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:00.513494 16032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:00.517652 16032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:00.517710 16032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:00.517803 16032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:00.523025 16032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36997
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:00.524094 16032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:00.525187 16032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:00.534822 16040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:00.534793 16037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:00.534999 16032 server_base.cc:1034] running on GCE node
W20260923 18:00:00.534786 16038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:00.535465 16032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:00.536170 16032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:00.537331 16032 hybrid_clock.cc:648] HybridClock initialized: now 1790186400537316 us; error 29 us; skew 500 ppm
Sep 23 18:00:00 dist-test-slave-vr57 krb5kdc[15925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186400, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:00.540102 16032 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:00:00.540870 16032 webserver.cc:460] Webserver started at http://127.15.139.129:35335/ using document root <none> and password file <none>
I20260923 18:00:00.541385 16032 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:00.541442 16032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:00.541605 16032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:00.543658 16032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "1e3d1be34fc748598af2dad280df604a"
format_stamp: "Formatted at 2026-09-23 18:00:00 on dist-test-slave-vr57"
server_key: "f150c9088520b6f83e14822629b18d8b"
server_key_iv: "24bfc8d160ff25072e5cf4e726fa8d60"
server_key_version: "encryptionkey@0"
I20260923 18:00:00.544243 16032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "1e3d1be34fc748598af2dad280df604a"
format_stamp: "Formatted at 2026-09-23 18:00:00 on dist-test-slave-vr57"
server_key: "f150c9088520b6f83e14822629b18d8b"
server_key_iv: "24bfc8d160ff25072e5cf4e726fa8d60"
server_key_version: "encryptionkey@0"
I20260923 18:00:00.547847 16032 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260923 18:00:00.550194 16047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:00.551438 16032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:00.551604 16032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "1e3d1be34fc748598af2dad280df604a"
format_stamp: "Formatted at 2026-09-23 18:00:00 on dist-test-slave-vr57"
server_key: "f150c9088520b6f83e14822629b18d8b"
server_key_iv: "24bfc8d160ff25072e5cf4e726fa8d60"
server_key_version: "encryptionkey@0"
I20260923 18:00:00.551719 16032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:00.567162 16032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:00.570561 16032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:00.570776 16032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:00.571373 16032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:00.572270 16032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:00.572342 16032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:00.572409 16032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:00.572461 16032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:00.581718 16032 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:36037
I20260923 18:00:00.581740 16160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:36037 every 8 connection(s)
I20260923 18:00:00.582602 16032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:00.585879 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 16032
I20260923 18:00:00.586010 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:00.586470 15918 external_mini_cluster.cc:1428] Setting key db7ae322af0a9cd2143ea80c039ba7a1
Sep 23 18:00:00 dist-test-slave-vr57 krb5kdc[15925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186400, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
I20260923 18:00:00.598166 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:00.584501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:41379 (local address 127.15.139.190:36997)
0923 18:00:00.584636 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:00.584640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:00.585422 (+ 782us) server_negotiation.cc:407] Connection header received
0923 18:00:00.587370 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:00.587373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:00.587423 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:00.587534 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:00.588953 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:00.589530 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:00.590723 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:00.590984 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:00.591102 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:00.594092 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:00.594098 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:00.594101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:00.594131 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:00.595994 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:00.596621 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:00.596624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:00.596625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:00.596703 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:00.597022 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:00.597024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:00.597025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:00.597203 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:00.597313 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:00.597890 (+ 577us) server_negotiation.cc:299] Negotiation successful
0923 18:00:00.598035 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:00:00.599143 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:00.584789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36997 (local address 127.15.139.129:41379)
0923 18:00:00.585257 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:00.585300 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:00.587075 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:00.587757 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:00.587770 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:00.588282 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:00.588799 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:00.588814 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:00.589699 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:00.589702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:00.590586 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:00.590597 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:00.591146 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:00.591150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:00.591336 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:00.592042 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:00.592059 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:00.593887 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:00.596129 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:00.596134 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:00.596147 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:00.596508 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:00.596792 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:00.596794 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:00.596795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:00.596941 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:00.597310 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:00.597316 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:00.597630 (+ 314us) client_negotiation.cc:769] Sending connection context
0923 18:00:00.597802 (+ 172us) client_negotiation.cc:240] Negotiation successful
0923 18:00:00.598114 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":124,"threads_started":1}
I20260923 18:00:00.600509 16161 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36997
I20260923 18:00:00.600747 16161 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:00.601205 16161 heartbeater.cc:511] Master 127.15.139.190:36997 requested a full tablet report, sending...
I20260923 18:00:00.602687 15970 ts_manager.cc:194] Registered new tserver with Master: 1e3d1be34fc748598af2dad280df604a (127.15.139.129:36037)
I20260923 18:00:00.603888 15970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:41379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:00:00.614401 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36997
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:00.723446 16166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:00.723798 16166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:00.723881 16166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:00.727902 16166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:00.727982 16166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:00.728070 16166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:00.733479 16166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36997
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:00.734647 16166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:00.735791 16166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:00.746826 16172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:00.746848 16171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:00.746837 16174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:00.747076 16166 server_base.cc:1034] running on GCE node
I20260923 18:00:00.747666 16166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:00.748435 16166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:00.749641 16166 hybrid_clock.cc:648] HybridClock initialized: now 1790186400749624 us; error 28 us; skew 500 ppm
Sep 23 18:00:00 dist-test-slave-vr57 krb5kdc[15925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186400, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:00.752666 16166 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:00:00.753547 16166 webserver.cc:460] Webserver started at http://127.15.139.130:45727/ using document root <none> and password file <none>
I20260923 18:00:00.754145 16166 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:00.754237 16166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:00.754531 16166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:00.756778 16166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "f75819151cf14b80bbec79ccc5ad5990"
format_stamp: "Formatted at 2026-09-23 18:00:00 on dist-test-slave-vr57"
server_key: "ff5be2800b19efeaad00f8dacdd9ae75"
server_key_iv: "0d37d63c68f89840ef2b845a4f36a480"
server_key_version: "encryptionkey@0"
I20260923 18:00:00.757433 16166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "f75819151cf14b80bbec79ccc5ad5990"
format_stamp: "Formatted at 2026-09-23 18:00:00 on dist-test-slave-vr57"
server_key: "ff5be2800b19efeaad00f8dacdd9ae75"
server_key_iv: "0d37d63c68f89840ef2b845a4f36a480"
server_key_version: "encryptionkey@0"
I20260923 18:00:00.761175 16166 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260923 18:00:00.763703 16181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:00.764911 16166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:00:00.765051 16166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "f75819151cf14b80bbec79ccc5ad5990"
format_stamp: "Formatted at 2026-09-23 18:00:00 on dist-test-slave-vr57"
server_key: "ff5be2800b19efeaad00f8dacdd9ae75"
server_key_iv: "0d37d63c68f89840ef2b845a4f36a480"
server_key_version: "encryptionkey@0"
I20260923 18:00:00.765161 16166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:00.779006 16166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:00.782117 16166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:00.782326 16166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:00.782917 16166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:00.783829 16166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:00.783900 16166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:00.783972 16166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:00.784025 16166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:00.793334 16166 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:35609
I20260923 18:00:00.793358 16294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:35609 every 8 connection(s)
I20260923 18:00:00.794216 16166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:00.800513 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 16166
I20260923 18:00:00.800606 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:00.800868 15918 external_mini_cluster.cc:1428] Setting key d571c8aa2133c5c0872ad2f0e7f3845f
Sep 23 18:00:00 dist-test-slave-vr57 krb5kdc[15925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186400, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:00.808674 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:00.796018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:41963 (local address 127.15.139.190:36997)
0923 18:00:00.796146 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:00.796149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:00.796923 (+ 774us) server_negotiation.cc:407] Connection header received
0923 18:00:00.798230 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:00.798233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:00.798289 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:00.798398 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:00.799651 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:00.800189 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:00.801585 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:00.801829 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:00.801940 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:00.804579 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:00.804584 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:00.804588 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:00.804621 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:00.806330 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:00.806849 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:00.806855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:00.806860 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:00.806956 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:00.807297 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:00.807303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:00.807309 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:00.807651 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:00.807773 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:00.808311 (+ 538us) server_negotiation.cc:299] Negotiation successful
0923 18:00:00.808442 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":44}
I20260923 18:00:00.809464 16297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:00.796352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36997 (local address 127.15.139.130:41963)
0923 18:00:00.796777 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:00.796819 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:00.798020 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:00.798583 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:00.798591 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:00.798943 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:00.799446 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:00.799461 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:00.800311 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:00.800315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:00.801466 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:00.801476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:00.801993 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:00.801998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:00.802171 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:00.802784 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:00.802800 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:00.804398 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:00.806464 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:00.806469 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:00.806480 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:00.806744 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:00.807065 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:00.807067 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:00.807068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:00.807199 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:00.807795 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:00.807801 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:00.808062 (+ 261us) client_negotiation.cc:769] Sending connection context
0923 18:00:00.808232 (+ 170us) client_negotiation.cc:240] Negotiation successful
0923 18:00:00.808503 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
I20260923 18:00:00.810829 16295 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36997
I20260923 18:00:00.811069 16295 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:00.811645 16295 heartbeater.cc:511] Master 127.15.139.190:36997 requested a full tablet report, sending...
I20260923 18:00:00.812667 15970 ts_manager.cc:194] Registered new tserver with Master: f75819151cf14b80bbec79ccc5ad5990 (127.15.139.130:35609)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
I20260923 18:00:00.813525 15970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:41963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:00:00.829085 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36997
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:00.940678 16300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:00.940970 16300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:00.941044 16300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:00.945331 16300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:00.945415 16300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:00.945508 16300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:00.950716 16300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36997
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:00.951824 16300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:00.952838 16300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:00.962561 16308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:00.962566 16306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:00.962963 16300 server_base.cc:1034] running on GCE node
W20260923 18:00:00.962556 16305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:00.963436 16300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:00.964156 16300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:00.965318 16300 hybrid_clock.cc:648] HybridClock initialized: now 1790186400965303 us; error 30 us; skew 500 ppm
Sep 23 18:00:00 dist-test-slave-vr57 krb5kdc[15925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186400, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:00.968159 16300 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:00:00.969121 16300 webserver.cc:460] Webserver started at http://127.15.139.131:34021/ using document root <none> and password file <none>
I20260923 18:00:00.969689 16300 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:00.969750 16300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:00.969987 16300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:00.972152 16300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "9cdaf0fec4aa44c29b23d5c855da4762"
format_stamp: "Formatted at 2026-09-23 18:00:00 on dist-test-slave-vr57"
server_key: "0efd28a61e372a1b4fd4b698a69aeecb"
server_key_iv: "9c68ed0232d181c87ab44eb7bdd0a7be"
server_key_version: "encryptionkey@0"
I20260923 18:00:00.972723 16300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "9cdaf0fec4aa44c29b23d5c855da4762"
format_stamp: "Formatted at 2026-09-23 18:00:00 on dist-test-slave-vr57"
server_key: "0efd28a61e372a1b4fd4b698a69aeecb"
server_key_iv: "9c68ed0232d181c87ab44eb7bdd0a7be"
server_key_version: "encryptionkey@0"
I20260923 18:00:00.976578 16300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260923 18:00:00.980145 16315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:00.981839 16300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:00:00.981992 16300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "9cdaf0fec4aa44c29b23d5c855da4762"
format_stamp: "Formatted at 2026-09-23 18:00:00 on dist-test-slave-vr57"
server_key: "0efd28a61e372a1b4fd4b698a69aeecb"
server_key_iv: "9c68ed0232d181c87ab44eb7bdd0a7be"
server_key_version: "encryptionkey@0"
I20260923 18:00:00.982132 16300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:00.997983 16300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:01.000927 16300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:01.001094 16300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:01.001613 16300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:01.002445 16300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:01.002494 16300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:01.002564 16300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:01.002605 16300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:01.011927 16300 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:34859
I20260923 18:00:01.011973 16428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:34859 every 8 connection(s)
I20260923 18:00:01.012974 16300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:01.015676 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 16300
I20260923 18:00:01.015807 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:00:01.016144 15918 external_mini_cluster.cc:1428] Setting key 24d7028c341d003165fe9cb28cb0c4e1
Sep 23 18:00:01 dist-test-slave-vr57 krb5kdc[15925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186400, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:01.026764 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:01.014778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:38013 (local address 127.15.139.190:36997)
0923 18:00:01.014924 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:01.014927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:01.015882 (+ 955us) server_negotiation.cc:407] Connection header received
0923 18:00:01.017623 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:01.017626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:01.017686 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:01.017770 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:01.019113 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:01.019705 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:01.020779 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:01.020982 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:01.021061 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:01.023448 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:01.023451 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:01.023453 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:01.023535 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:01.024798 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:01.025298 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:01.025301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:01.025302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:01.025376 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:01.025684 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:01.025686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:01.025687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:01.025850 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:01.025928 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:01.026503 (+ 575us) server_negotiation.cc:299] Negotiation successful
0923 18:00:01.026633 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 18:00:01.027577 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:01.015112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36997 (local address 127.15.139.131:38013)
0923 18:00:01.015693 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:01.015757 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:01.017381 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:01.017965 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:01.017972 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:01.018383 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:01.018984 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:01.018998 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:01.019808 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:01.019811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:01.020679 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:01.020687 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:01.021108 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:01.021111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:01.021248 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:01.021863 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:01.021879 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:01.023303 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:01.024903 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:01.024907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:01.024918 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:01.025155 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:01.025462 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:01.025464 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:01.025465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:01.025602 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:01.025983 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:01.025992 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:01.026222 (+ 230us) client_negotiation.cc:769] Sending connection context
0923 18:00:01.026429 (+ 207us) client_negotiation.cc:240] Negotiation successful
0923 18:00:01.026690 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":131,"threads_started":1}
I20260923 18:00:01.028746 16429 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36997
I20260923 18:00:01.029004 16429 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:01.029491 16429 heartbeater.cc:511] Master 127.15.139.190:36997 requested a full tablet report, sending...
I20260923 18:00:01.030537 15970 ts_manager.cc:194] Registered new tserver with Master: 9cdaf0fec4aa44c29b23d5c855da4762 (127.15.139.131:34859)
I20260923 18:00:01.031103 15970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:38013
I20260923 18:00:01.041992 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 18:00:01 dist-test-slave-vr57 krb5kdc[15925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186401, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 18:00:01 dist-test-slave-vr57 krb5kdc[15925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186401, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.139.129@KRBTEST.COM
I20260923 18:00:01.068504 16436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:01.058765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54054 (local address 127.15.139.129:36037)
0923 18:00:01.059082 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:01.059095 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:01.059120 (+ 25us) server_negotiation.cc:407] Connection header received
0923 18:00:01.059247 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:01.059252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:01.059441 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:01.059643 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:01.060328 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:01.061091 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:01.061890 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:01.062269 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:01.062352 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:01.064816 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:01.064824 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:01.064838 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:01.064863 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:01.066455 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:01.066974 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:01.066979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:01.066980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:01.067059 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:01.067332 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:01.067334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:01.067335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:01.067648 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:01.067904 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:01.068208 (+ 304us) server_negotiation.cc:299] Negotiation successful
0923 18:00:01.068333 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":99,"threads_started":1}
W20260923 18:00:01.069567 16075 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54054
Sep 23 18:00:01 dist-test-slave-vr57 krb5kdc[15925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 18:00:01 dist-test-slave-vr57 krb5kdc[15925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.129@KRBTEST.COM
I20260923 18:00:01.095934 16436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:01.087004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54064 (local address 127.15.139.129:36037)
0923 18:00:01.087163 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:01.087169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:01.087354 (+ 185us) server_negotiation.cc:407] Connection header received
0923 18:00:01.087567 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:01.087571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:01.087624 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:01.087712 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:01.088414 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:01.088931 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:01.089776 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:01.090010 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:01.090120 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:01.092575 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:01.092578 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:01.092580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:01.092605 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:01.094118 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:01.094599 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:01.094602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:01.094603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:01.094682 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:01.095017 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:01.095019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:01.095020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:01.095191 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:01.095295 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:01.095693 (+ 398us) server_negotiation.cc:299] Negotiation successful
0923 18:00:01.095807 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 18:00:01.097097 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 16032
I20260923 18:00:01.103209 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 16166
I20260923 18:00:01.111140 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 16300
I20260923 18:00:01.116778 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 15938
2026-09-23T18:00:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2245 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:01 dist-test-slave-vr57 krb5kdc[16443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:01 dist-test-slave-vr57 krb5kdc[16443](info): set up 2 sockets
Sep 23 18:00:01 dist-test-slave-vr57 krb5kdc[16443](info): commencing operation
krb5kdc: starting...
W20260923 18:00:02.159750 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:02 dist-test-slave-vr57 krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:00:02.287570 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40575
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:40575
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:02.403355 16456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:02.403746 16456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:02.403827 16456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:02.408170 16456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:02.408272 16456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:02.408309 16456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:02.408344 16456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:02.408380 16456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:02.414161 16456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:40575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:02.415364 16456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:02.416545 16456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:02.425287 16464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:02.425352 16462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:02.425329 16456 server_base.cc:1034] running on GCE node
W20260923 18:00:02.425283 16461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:02.426059 16456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:02.427296 16456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:02.428506 16456 hybrid_clock.cc:648] HybridClock initialized: now 1790186402428488 us; error 41 us; skew 500 ppm
Sep 23 18:00:02 dist-test-slave-vr57 krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186402, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:02.431380 16456 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:00:02.432307 16456 webserver.cc:460] Webserver started at http://127.15.139.190:35917/ using document root <none> and password file <none>
I20260923 18:00:02.432834 16456 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:02.432895 16456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:02.433058 16456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:02.435243 16456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "540d6dbc17cd4b8f82f367aa8a07a11c"
format_stamp: "Formatted at 2026-09-23 18:00:02 on dist-test-slave-vr57"
server_key: "75eded08146a1a98766fe8146c1ac347"
server_key_iv: "71e0968e80eb8b71cd3065f1ac822b18"
server_key_version: "encryptionkey@0"
I20260923 18:00:02.435830 16456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "540d6dbc17cd4b8f82f367aa8a07a11c"
format_stamp: "Formatted at 2026-09-23 18:00:02 on dist-test-slave-vr57"
server_key: "75eded08146a1a98766fe8146c1ac347"
server_key_iv: "71e0968e80eb8b71cd3065f1ac822b18"
server_key_version: "encryptionkey@0"
I20260923 18:00:02.439479 16456 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:02.441915 16471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:02.443223 16456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260923 18:00:02.443331 16456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "540d6dbc17cd4b8f82f367aa8a07a11c"
format_stamp: "Formatted at 2026-09-23 18:00:02 on dist-test-slave-vr57"
server_key: "75eded08146a1a98766fe8146c1ac347"
server_key_iv: "71e0968e80eb8b71cd3065f1ac822b18"
server_key_version: "encryptionkey@0"
I20260923 18:00:02.443480 16456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:02.464046 16456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:02.467484 16456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:02.467684 16456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:02.475189 16456 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:40575
I20260923 18:00:02.475206 16523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:40575 every 8 connection(s)
I20260923 18:00:02.476426 16456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:02.479835 16524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:02.484332 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 16456
I20260923 18:00:02.484429 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:02.484705 15918 external_mini_cluster.cc:1428] Setting key 5fc7c7223e4030b25c45c23e4630e96d
I20260923 18:00:02.486956 16524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c: Bootstrap starting.
I20260923 18:00:02.490149 16524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:02.491148 16524 log.cc:824] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:02.493443 16524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c: No bootstrap required, opened a new log
Sep 23 18:00:02 dist-test-slave-vr57 krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:02.498167 16524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d6dbc17cd4b8f82f367aa8a07a11c" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40575 } }
I20260923 18:00:02.498456 16524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:02.498524 16524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 540d6dbc17cd4b8f82f367aa8a07a11c, State: Initialized, Role: FOLLOWER
I20260923 18:00:02.499051 16524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d6dbc17cd4b8f82f367aa8a07a11c" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40575 } }
I20260923 18:00:02.499220 16524 raft_consensus.cc:388] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:02.499333 16524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:02.499476 16524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:02.500005 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:02.486229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46822 (local address 127.15.139.190:40575)
0923 18:00:02.486688 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:02.486709 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:02.486744 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:00:02.487456 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:02.487476 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:02.487884 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:02.488235 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:02.489065 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:02.490524 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:02.491423 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:02.491984 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:02.492169 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:02.494814 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:02.494819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:02.494834 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:02.494864 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:02.496728 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:02.497205 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:02.497210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:02.497215 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:02.497332 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:02.497625 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:02.497629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:02.497631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:02.498058 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:02.498273 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:02.498597 (+ 324us) server_negotiation.cc:299] Negotiation successful
0923 18:00:02.498945 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":168,"threads_started":1}
I20260923 18:00:02.567708 16524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d6dbc17cd4b8f82f367aa8a07a11c" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40575 } }
I20260923 18:00:02.568343 16524 leader_election.cc:302] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 540d6dbc17cd4b8f82f367aa8a07a11c; no voters:
I20260923 18:00:02.568816 16524 leader_election.cc:288] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:02.568984 16529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:02.569226 16529 raft_consensus.cc:686] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [term 1 LEADER]: Becoming Leader. State: Replica: 540d6dbc17cd4b8f82f367aa8a07a11c, State: Running, Role: LEADER
I20260923 18:00:02.569617 16529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d6dbc17cd4b8f82f367aa8a07a11c" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40575 } }
I20260923 18:00:02.570295 16524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:02.571616 16530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "540d6dbc17cd4b8f82f367aa8a07a11c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d6dbc17cd4b8f82f367aa8a07a11c" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40575 } } }
I20260923 18:00:02.571787 16530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:02.572114 16531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 540d6dbc17cd4b8f82f367aa8a07a11c. Latest consensus state: current_term: 1 leader_uuid: "540d6dbc17cd4b8f82f367aa8a07a11c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d6dbc17cd4b8f82f367aa8a07a11c" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40575 } } }
I20260923 18:00:02.572221 16531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:02.572546 16538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:02.576069 16538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:02.582633 16538 catalog_manager.cc:1347] Generated new cluster ID: 2b5a37001c154266baf37495aa72ad46
I20260923 18:00:02.582720 16538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:02.594676 16538 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:02.596199 16538 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:02.608057 16538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 540d6dbc17cd4b8f82f367aa8a07a11c: Generated new TSK 0
I20260923 18:00:02.608912 16538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
W20260923 18:00:03.670428 15918 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.139.129: real 1.054s user 0.000s sys 0.004s
I20260923 18:00:03.671066 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40575
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:03.789062 16550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:03.789345 16550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:03.789393 16550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:03.793503 16550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:03.793573 16550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:03.793634 16550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:03.799046 16550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40575
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:03.800235 16550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:03.801226 16550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:03.811465 16558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:03.811515 16556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:03.811465 16555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:03.811812 16550 server_base.cc:1034] running on GCE node
I20260923 18:00:03.812258 16550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:03.812893 16550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:03.814206 16550 hybrid_clock.cc:648] HybridClock initialized: now 1790186403814190 us; error 26 us; skew 500 ppm
Sep 23 18:00:03 dist-test-slave-vr57 krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186403, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:03.824219 16550 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:00:03.825268 16550 webserver.cc:460] Webserver started at http://127.15.139.129:44883/ using document root <none> and password file <none>
I20260923 18:00:03.826072 16550 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:03.826169 16550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:03.826390 16550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:03.828568 16550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "3e710cfbf08645118fc7cf171f63a1f9"
format_stamp: "Formatted at 2026-09-23 18:00:03 on dist-test-slave-vr57"
server_key: "be153e7539c8847c771076297b2de364"
server_key_iv: "8bd8f28c77511ca34b7cf63c9f67af28"
server_key_version: "encryptionkey@0"
I20260923 18:00:03.829103 16550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "3e710cfbf08645118fc7cf171f63a1f9"
format_stamp: "Formatted at 2026-09-23 18:00:03 on dist-test-slave-vr57"
server_key: "be153e7539c8847c771076297b2de364"
server_key_iv: "8bd8f28c77511ca34b7cf63c9f67af28"
server_key_version: "encryptionkey@0"
I20260923 18:00:03.833137 16550 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:00:03.836041 16565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:03.837513 16550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:03.837622 16550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "3e710cfbf08645118fc7cf171f63a1f9"
format_stamp: "Formatted at 2026-09-23 18:00:03 on dist-test-slave-vr57"
server_key: "be153e7539c8847c771076297b2de364"
server_key_iv: "8bd8f28c77511ca34b7cf63c9f67af28"
server_key_version: "encryptionkey@0"
I20260923 18:00:03.837718 16550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:03.852210 16550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:03.855324 16550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:03.855484 16550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:03.856099 16550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:03.856863 16550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:03.856907 16550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:03.856947 16550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:03.856962 16550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:03.865866 16550 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:38019
I20260923 18:00:03.865892 16678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:38019 every 8 connection(s)
I20260923 18:00:03.866788 16550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:03.868177 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 16550
I20260923 18:00:03.868291 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:03.868605 15918 external_mini_cluster.cc:1428] Setting key 943f145f13e2ae565d3a5c035107c94e
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
Sep 23 18:00:03 dist-test-slave-vr57 krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186403, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:03.884181 16681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:03.869064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:48295 (local address 127.15.139.190:40575)
0923 18:00:03.869321 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:03.869325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:03.870230 (+ 905us) server_negotiation.cc:407] Connection header received
0923 18:00:03.872357 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:03.872362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:03.872428 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:03.872525 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:03.874481 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:03.875052 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:03.876975 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:03.877211 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:03.877293 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:03.880297 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:03.880301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:03.880304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:03.880336 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:03.881864 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:03.882429 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:03.882432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:03.882434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:03.882516 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:03.882861 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:03.882873 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:03.882874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:03.883068 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:03.883178 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:03.883827 (+ 649us) server_negotiation.cc:299] Negotiation successful
0923 18:00:03.883973 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":105,"threads_started":1}
I20260923 18:00:03.884974 16682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:03.869521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40575 (local address 127.15.139.129:48295)
0923 18:00:03.870059 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:03.870145 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:03.872085 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:03.873090 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:03.873097 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:03.873613 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:03.874303 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:03.874325 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:03.875186 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:03.875192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:03.876846 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:03.876861 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:03.877396 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:03.877401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:03.877603 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:03.878241 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:03.878266 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:03.880115 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:03.882035 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:03.882041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:03.882057 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:03.882316 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:03.882612 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:03.882615 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:03.882617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:03.882757 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:03.883175 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:03.883183 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:03.883479 (+ 296us) client_negotiation.cc:769] Sending connection context
0923 18:00:03.883774 (+ 295us) client_negotiation.cc:240] Negotiation successful
0923 18:00:03.884121 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":144,"threads_started":1}
I20260923 18:00:03.886530 16679 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40575
I20260923 18:00:03.886793 16679 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:03.887305 16679 heartbeater.cc:511] Master 127.15.139.190:40575 requested a full tablet report, sending...
I20260923 18:00:03.889099 16488 ts_manager.cc:194] Registered new tserver with Master: 3e710cfbf08645118fc7cf171f63a1f9 (127.15.139.129:38019)
I20260923 18:00:03.890663 16488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:48295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:00:04.893446 16679 heartbeater.cc:503] Master 127.15.139.190:40575 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
W20260923 18:00:05.217197 15918 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.139.130: real 1.346s user 0.000s sys 0.004s
I20260923 18:00:05.217763 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40575
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:05.349702 16685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:05.350063 16685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:05.350116 16685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:05.355108 16685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:05.355188 16685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:05.355259 16685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:05.361231 16685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40575
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:05.362425 16685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:05.363600 16685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:05.374051 16691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:05.374056 16693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:05.374056 16690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:05.374550 16685 server_base.cc:1034] running on GCE node
I20260923 18:00:05.374917 16685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:05.375689 16685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:05.376895 16685 hybrid_clock.cc:648] HybridClock initialized: now 1790186405376859 us; error 43 us; skew 500 ppm
Sep 23 18:00:05 dist-test-slave-vr57 krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186405, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:05.380483 16685 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:00:05.381580 16685 webserver.cc:460] Webserver started at http://127.15.139.130:38571/ using document root <none> and password file <none>
I20260923 18:00:05.382282 16685 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:05.382398 16685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:05.382690 16685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:05.385212 16685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "1bcfaf6031394fdda45138d8250ebc64"
format_stamp: "Formatted at 2026-09-23 18:00:05 on dist-test-slave-vr57"
server_key: "544cdfe2bfebb2e4635edebe02221669"
server_key_iv: "825b5c5dc2347d2986d3ee64e6c50962"
server_key_version: "encryptionkey@0"
I20260923 18:00:05.385824 16685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "1bcfaf6031394fdda45138d8250ebc64"
format_stamp: "Formatted at 2026-09-23 18:00:05 on dist-test-slave-vr57"
server_key: "544cdfe2bfebb2e4635edebe02221669"
server_key_iv: "825b5c5dc2347d2986d3ee64e6c50962"
server_key_version: "encryptionkey@0"
I20260923 18:00:05.390314 16685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:00:05.393188 16700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:05.394599 16685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:00:05.394762 16685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "1bcfaf6031394fdda45138d8250ebc64"
format_stamp: "Formatted at 2026-09-23 18:00:05 on dist-test-slave-vr57"
server_key: "544cdfe2bfebb2e4635edebe02221669"
server_key_iv: "825b5c5dc2347d2986d3ee64e6c50962"
server_key_version: "encryptionkey@0"
I20260923 18:00:05.394876 16685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:05.407711 16685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:05.411468 16685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:05.411743 16685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:05.412423 16685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:05.413338 16685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:05.413411 16685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:05.413489 16685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:05.413537 16685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:05.423615 16685 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:42941
I20260923 18:00:05.423650 16813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:42941 every 8 connection(s)
I20260923 18:00:05.424685 16685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:05.425402 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 16685
I20260923 18:00:05.425526 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:05.425822 15918 external_mini_cluster.cc:1428] Setting key 7e66f5c895c198ce4974f49428083c43
Sep 23 18:00:05 dist-test-slave-vr57 krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186405, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
I20260923 18:00:05.441118 16816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:05.426728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:56781 (local address 127.15.139.190:40575)
0923 18:00:05.427375 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:05.427382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:05.427971 (+ 589us) server_negotiation.cc:407] Connection header received
0923 18:00:05.429695 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:05.429697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:05.429744 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:05.429827 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:05.431282 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:05.431891 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:05.433967 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:05.434197 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:05.434276 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:05.437473 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:05.437476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:05.437478 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:05.437507 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:05.438977 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:05.439525 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:05.439530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:05.439534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:05.439624 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:05.439972 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:05.439975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:05.439977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:05.440159 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:05.440277 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:05.440797 (+ 520us) server_negotiation.cc:299] Negotiation successful
0923 18:00:05.440927 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":85,"threads_started":1}
I20260923 18:00:05.441957 16817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:05.427160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40575 (local address 127.15.139.130:56781)
0923 18:00:05.427822 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:05.427867 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:05.429415 (+ 1548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:05.430042 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:05.430054 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:05.430610 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:05.431136 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:05.431152 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:05.432029 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:05.432036 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:05.433813 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:05.433828 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:05.434370 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:05.434378 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:05.434669 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:05.435377 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:05.435420 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:05.437291 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:05.439097 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:05.439104 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:05.439120 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:05.439385 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:05.439726 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:05.439729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:05.439731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:05.439867 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:05.440276 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:05.440283 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:05.440549 (+ 266us) client_negotiation.cc:769] Sending connection context
0923 18:00:05.440749 (+ 200us) client_negotiation.cc:240] Negotiation successful
0923 18:00:05.441080 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":149,"threads_started":1}
I20260923 18:00:05.443238 16814 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40575
I20260923 18:00:05.443483 16814 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:05.444010 16814 heartbeater.cc:511] Master 127.15.139.190:40575 requested a full tablet report, sending...
I20260923 18:00:05.445050 16488 ts_manager.cc:194] Registered new tserver with Master: 1bcfaf6031394fdda45138d8250ebc64 (127.15.139.130:42941)
I20260923 18:00:05.445732 16488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:56781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
W20260923 18:00:06.077565 15918 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.139.131: real 0.650s user 0.002s sys 0.001s
I20260923 18:00:06.078279 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40575
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:06.199120 16820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:06.199424 16820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:06.199565 16820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:06.204051 16820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:06.204150 16820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:06.204252 16820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:06.210042 16820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40575
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:06.211323 16820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:06.212399 16820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:06.222711 16825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:06.222699 16826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:06.222865 16820 server_base.cc:1034] running on GCE node
W20260923 18:00:06.223097 16828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:06.223542 16820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:06.224231 16820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:06.225425 16820 hybrid_clock.cc:648] HybridClock initialized: now 1790186406225402 us; error 37 us; skew 500 ppm
Sep 23 18:00:06 dist-test-slave-vr57 krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186406, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:06.228500 16820 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:00:06.229487 16820 webserver.cc:460] Webserver started at http://127.15.139.131:46559/ using document root <none> and password file <none>
I20260923 18:00:06.230119 16820 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:06.230188 16820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:06.230435 16820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:06.232919 16820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "aaccc304fad74194b27a8158864ba8d4"
format_stamp: "Formatted at 2026-09-23 18:00:06 on dist-test-slave-vr57"
server_key: "3b89965ca99291bc6eb7c8169afc1cee"
server_key_iv: "b9ce517436f7962ec664ee86c1fc24c2"
server_key_version: "encryptionkey@0"
I20260923 18:00:06.233517 16820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "aaccc304fad74194b27a8158864ba8d4"
format_stamp: "Formatted at 2026-09-23 18:00:06 on dist-test-slave-vr57"
server_key: "3b89965ca99291bc6eb7c8169afc1cee"
server_key_iv: "b9ce517436f7962ec664ee86c1fc24c2"
server_key_version: "encryptionkey@0"
I20260923 18:00:06.238083 16820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:06.241645 16835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:06.243371 16820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:06.243587 16820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "aaccc304fad74194b27a8158864ba8d4"
format_stamp: "Formatted at 2026-09-23 18:00:06 on dist-test-slave-vr57"
server_key: "3b89965ca99291bc6eb7c8169afc1cee"
server_key_iv: "b9ce517436f7962ec664ee86c1fc24c2"
server_key_version: "encryptionkey@0"
I20260923 18:00:06.243763 16820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:06.258227 16820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:06.262228 16820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:06.262465 16820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:06.263252 16820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:06.264237 16820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:06.264288 16820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:06.264331 16820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:06.264348 16820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:06.274420 16820 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:33441
I20260923 18:00:06.274519 16948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:33441 every 8 connection(s)
I20260923 18:00:06.275494 16820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:06.275777 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 16820
I20260923 18:00:06.275923 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:00:06.276244 15918 external_mini_cluster.cc:1428] Setting key 11a3bc7683b8bb96449de23cb0d636c4
Sep 23 18:00:06 dist-test-slave-vr57 krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186406, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:06.291078 16951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.277659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:49941 (local address 127.15.139.190:40575)
0923 18:00:06.279232 (+ 1573us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:06.279237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:06.279254 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:00:06.280343 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:06.280347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:06.280402 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:06.280491 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:06.281792 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.282352 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.283608 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.283852 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.283969 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.286791 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:06.286798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:06.286805 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:06.286845 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:06.288813 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:06.289429 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:06.289433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:06.289435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:06.289524 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:06.289910 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:06.289913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:06.289915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:06.290112 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:06.290218 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:06.290764 (+ 546us) server_negotiation.cc:299] Negotiation successful
0923 18:00:06.290898 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1488,"thread_start_us":63,"threads_started":1}
I20260923 18:00:06.292088 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.278152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40575 (local address 127.15.139.131:49941)
0923 18:00:06.278658 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:06.278714 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:06.280116 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:06.280697 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:06.280704 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:06.281096 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:06.281639 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:06.281654 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.282504 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.282507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:06.283423 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:06.283431 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.284013 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.284017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:06.284210 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.284850 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:06.284867 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:06.286586 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:06.288998 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:06.289003 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:06.289018 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:06.289296 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:06.289621 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:06.289626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:06.289628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:06.289773 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:06.290228 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:06.290236 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:06.290524 (+ 288us) client_negotiation.cc:769] Sending connection context
0923 18:00:06.290724 (+ 200us) client_negotiation.cc:240] Negotiation successful
0923 18:00:06.291096 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":147,"threads_started":1}
I20260923 18:00:06.293236 16949 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40575
I20260923 18:00:06.293571 16949 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:06.294165 16949 heartbeater.cc:511] Master 127.15.139.190:40575 requested a full tablet report, sending...
I20260923 18:00:06.295262 16488 ts_manager.cc:194] Registered new tserver with Master: aaccc304fad74194b27a8158864ba8d4 (127.15.139.131:33441)
I20260923 18:00:06.296277 16488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:49941
I20260923 18:00:06.302733 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:00:06.317142 16951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.307617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45722 (local address 127.15.139.190:40575)
0923 18:00:06.307808 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:06.307814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:06.307998 (+ 184us) server_negotiation.cc:407] Connection header received
0923 18:00:06.308236 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:06.308240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:06.308309 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:06.308434 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:06.309248 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.310001 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.311210 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.311479 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.311683 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.313338 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:06.313341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:06.313344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:06.313373 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:06.315070 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:06.315625 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:06.315632 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:06.315639 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:06.315729 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:06.316045 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:06.316055 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:06.316057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:06.316335 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:06.316523 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:06.316783 (+ 260us) server_negotiation.cc:299] Negotiation successful
0923 18:00:06.316934 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 18:00:06.322971 16488 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:00:06.326665 16488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:06.342965 16966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.336542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:33441 (local address 127.0.0.1:46450)
0923 18:00:06.337591 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:06.337605 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:06.337763 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:06.338445 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:06.338447 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:06.338467 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:06.338554 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:06.338559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.339675 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.339679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:06.340972 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:06.340982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.342335 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.342340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:06.342466 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.342491 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 18:00:06.342588 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 18:00:06.342771 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"mutex_wait_us":202,"thread_start_us":95,"threads_started":1}
I20260923 18:00:06.343176 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.336535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:42941 (local address 127.0.0.1:58904)
0923 18:00:06.337417 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:06.337477 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:06.337618 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:06.338250 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:06.338255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:06.338278 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:06.338424 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:06.338433 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.339961 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.339965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:06.341188 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:06.341199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.342210 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.342215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:06.342338 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.342436 (+ 98us) client_negotiation.cc:769] Sending connection context
0923 18:00:06.342643 (+ 207us) client_negotiation.cc:240] Negotiation successful
0923 18:00:06.342742 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":26,"thread_start_us":105,"threads_started":1}
I20260923 18:00:06.342965 16963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.337323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:38019 (local address 127.0.0.1:34152)
0923 18:00:06.337909 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:06.337922 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:06.338007 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:06.338696 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:06.338699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:06.338721 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:06.338798 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:06.338804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.340249 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.340252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:06.341372 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:06.341378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.342371 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.342376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:06.342492 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.342508 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 18:00:06.342588 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 18:00:06.342771 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":40,"thread_start_us":89,"threads_started":1}
I20260923 18:00:06.343091 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.336684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.15.139.130:42941)
0923 18:00:06.337133 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:06.337141 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:06.337767 (+ 626us) server_negotiation.cc:407] Connection header received
0923 18:00:06.337863 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:06.337869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:06.338111 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:06.338265 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:06.338555 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.339805 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.341313 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.342086 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.342204 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.342303 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:06.342844 (+ 541us) server_negotiation.cc:299] Negotiation successful
0923 18:00:06.342954 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":112,"threads_started":1}
I20260923 18:00:06.343883 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.337524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34152 (local address 127.15.139.129:38019)
0923 18:00:06.338141 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:06.338148 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:06.338168 (+ 20us) server_negotiation.cc:407] Connection header received
0923 18:00:06.338252 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:06.338261 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:06.338437 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:06.338604 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:06.338919 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.340122 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.341545 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.342250 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.342364 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.342522 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:06.343596 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0923 18:00:06.343703 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":119,"threads_started":1}
I20260923 18:00:06.344055 16965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.336774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46450 (local address 127.15.139.131:33441)
0923 18:00:06.337144 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:06.337156 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:06.337689 (+ 533us) server_negotiation.cc:407] Connection header received
0923 18:00:06.337998 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:06.338005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:06.338206 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:06.338330 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:06.338666 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.339544 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.341092 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.341935 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.342098 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.342264 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:06.342626 (+ 362us) server_negotiation.cc:299] Negotiation successful
0923 18:00:06.342742 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":94,"threads_started":1}
I20260923 18:00:06.346477 16748 tablet_service.cc:1467] Processing CreateTablet for tablet f814850951744882965409a75fe0bc3b (DEFAULT_TABLE table=test-table [id=7f309f672a034b12b150a036df0b9161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:06.346415 16883 tablet_service.cc:1467] Processing CreateTablet for tablet f814850951744882965409a75fe0bc3b (DEFAULT_TABLE table=test-table [id=7f309f672a034b12b150a036df0b9161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:06.347554 16748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f814850951744882965409a75fe0bc3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:06.347586 16883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f814850951744882965409a75fe0bc3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:06.353936 16968 tablet_bootstrap.cc:492] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64: Bootstrap starting.
I20260923 18:00:06.354576 16613 tablet_service.cc:1467] Processing CreateTablet for tablet f814850951744882965409a75fe0bc3b (DEFAULT_TABLE table=test-table [id=7f309f672a034b12b150a036df0b9161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:06.355206 16969 tablet_bootstrap.cc:492] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4: Bootstrap starting.
I20260923 18:00:06.355760 16613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f814850951744882965409a75fe0bc3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:06.356657 16968 tablet_bootstrap.cc:654] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:06.357524 16968 log.cc:824] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:06.357963 16969 tablet_bootstrap.cc:654] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:06.358860 16969 log.cc:824] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:06.360313 16968 tablet_bootstrap.cc:492] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64: No bootstrap required, opened a new log
I20260923 18:00:06.360517 16968 ts_tablet_manager.cc:1397] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260923 18:00:06.360528 16969 tablet_bootstrap.cc:492] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4: No bootstrap required, opened a new log
I20260923 18:00:06.360723 16969 ts_tablet_manager.cc:1397] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260923 18:00:06.362916 16972 tablet_bootstrap.cc:492] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9: Bootstrap starting.
I20260923 18:00:06.363888 16969 raft_consensus.cc:348] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcfaf6031394fdda45138d8250ebc64" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42941 } } peers { permanent_uuid: "aaccc304fad74194b27a8158864ba8d4" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 33441 } } peers { permanent_uuid: "3e710cfbf08645118fc7cf171f63a1f9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38019 } }
I20260923 18:00:06.364171 16969 raft_consensus.cc:374] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:06.364228 16969 raft_consensus.cc:729] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaccc304fad74194b27a8158864ba8d4, State: Initialized, Role: FOLLOWER
I20260923 18:00:06.364661 16969 consensus_queue.cc:260] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcfaf6031394fdda45138d8250ebc64" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42941 } } peers { permanent_uuid: "aaccc304fad74194b27a8158864ba8d4" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 33441 } } peers { permanent_uuid: "3e710cfbf08645118fc7cf171f63a1f9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38019 } }
I20260923 18:00:06.364861 16968 raft_consensus.cc:348] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcfaf6031394fdda45138d8250ebc64" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42941 } } peers { permanent_uuid: "aaccc304fad74194b27a8158864ba8d4" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 33441 } } peers { permanent_uuid: "3e710cfbf08645118fc7cf171f63a1f9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38019 } }
I20260923 18:00:06.365113 16968 raft_consensus.cc:374] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:06.365172 16968 raft_consensus.cc:729] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcfaf6031394fdda45138d8250ebc64, State: Initialized, Role: FOLLOWER
I20260923 18:00:06.365540 16972 tablet_bootstrap.cc:654] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:06.365698 16969 ts_tablet_manager.cc:1428] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260923 18:00:06.365634 16968 consensus_queue.cc:260] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcfaf6031394fdda45138d8250ebc64" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42941 } } peers { permanent_uuid: "aaccc304fad74194b27a8158864ba8d4" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 33441 } } peers { permanent_uuid: "3e710cfbf08645118fc7cf171f63a1f9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38019 } }
I20260923 18:00:06.366057 16949 heartbeater.cc:503] Master 127.15.139.190:40575 was elected leader, sending a full tablet report...
I20260923 18:00:06.366281 16814 heartbeater.cc:503] Master 127.15.139.190:40575 was elected leader, sending a full tablet report...
I20260923 18:00:06.366350 16972 log.cc:824] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:06.366591 16968 ts_tablet_manager.cc:1428] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260923 18:00:06.368569 16972 tablet_bootstrap.cc:492] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9: No bootstrap required, opened a new log
I20260923 18:00:06.368772 16972 ts_tablet_manager.cc:1397] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260923 18:00:06.373164 16972 raft_consensus.cc:348] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcfaf6031394fdda45138d8250ebc64" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42941 } } peers { permanent_uuid: "aaccc304fad74194b27a8158864ba8d4" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 33441 } } peers { permanent_uuid: "3e710cfbf08645118fc7cf171f63a1f9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38019 } }
I20260923 18:00:06.373420 16972 raft_consensus.cc:374] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:06.373484 16972 raft_consensus.cc:729] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e710cfbf08645118fc7cf171f63a1f9, State: Initialized, Role: FOLLOWER
I20260923 18:00:06.373963 16972 consensus_queue.cc:260] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcfaf6031394fdda45138d8250ebc64" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42941 } } peers { permanent_uuid: "aaccc304fad74194b27a8158864ba8d4" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 33441 } } peers { permanent_uuid: "3e710cfbf08645118fc7cf171f63a1f9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38019 } }
I20260923 18:00:06.374975 16972 ts_tablet_manager.cc:1428] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260923 18:00:06.426451 16815 tablet.cc:2369] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:00:06.451524 16976 raft_consensus.cc:482] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:06.451727 16976 raft_consensus.cc:504] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcfaf6031394fdda45138d8250ebc64" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42941 } } peers { permanent_uuid: "aaccc304fad74194b27a8158864ba8d4" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 33441 } } peers { permanent_uuid: "3e710cfbf08645118fc7cf171f63a1f9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38019 } }
I20260923 18:00:06.453008 16976 leader_election.cc:288] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcfaf6031394fdda45138d8250ebc64 (127.15.139.130:42941), aaccc304fad74194b27a8158864ba8d4 (127.15.139.131:33441)
I20260923 18:00:06.459357 16977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.453379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:33441 (local address 127.15.139.129:36261)
0923 18:00:06.453801 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:06.453818 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:06.453946 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:06.454302 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:06.454305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:06.454322 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:06.454513 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:06.454528 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.455867 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.455871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:06.457581 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:06.457594 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.458761 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.458765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:06.458949 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.458976 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 18:00:06.459069 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 18:00:06.459157 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":171,"thread_start_us":121,"threads_started":1}
I20260923 18:00:06.459453 16965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.453492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:36261 (local address 127.15.139.131:33441)
0923 18:00:06.453626 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:06.453630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:06.453926 (+ 296us) server_negotiation.cc:407] Connection header received
0923 18:00:06.454082 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:06.454085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:06.454138 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:06.454222 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:06.454715 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.455703 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.457751 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.458460 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.458624 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.458685 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:06.459120 (+ 435us) server_negotiation.cc:299] Negotiation successful
0923 18:00:06.459247 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 18:00:06.459997 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.454006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:60423 (local address 127.15.139.130:42941)
0923 18:00:06.454182 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:06.454187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:06.454203 (+ 16us) server_negotiation.cc:407] Connection header received
0923 18:00:06.454256 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:06.454258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:06.454298 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:06.454381 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:06.454715 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.455704 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.457751 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.458480 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.458624 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.458685 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:06.459742 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0923 18:00:06.459858 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 18:00:06.460394 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.453320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:42941 (local address 127.15.139.129:60423)
0923 18:00:06.453712 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:06.453724 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:06.453817 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:06.454422 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:06.454436 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:06.454449 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:06.454539 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:06.454543 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.455867 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.455871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:06.457581 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:06.457594 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.458761 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.458765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:06.458912 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.459602 (+ 690us) client_negotiation.cc:769] Sending connection context
0923 18:00:06.460154 (+ 552us) client_negotiation.cc:240] Negotiation successful
0923 18:00:06.460238 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":790,"thread_start_us":124,"threads_started":1}
I20260923 18:00:06.460376 16903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f814850951744882965409a75fe0bc3b" candidate_uuid: "3e710cfbf08645118fc7cf171f63a1f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaccc304fad74194b27a8158864ba8d4" is_pre_election: true
I20260923 18:00:06.460777 16903 raft_consensus.cc:2454] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e710cfbf08645118fc7cf171f63a1f9 in term 0.
I20260923 18:00:06.461091 16768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f814850951744882965409a75fe0bc3b" candidate_uuid: "3e710cfbf08645118fc7cf171f63a1f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcfaf6031394fdda45138d8250ebc64" is_pre_election: true
I20260923 18:00:06.461408 16768 raft_consensus.cc:2454] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e710cfbf08645118fc7cf171f63a1f9 in term 0.
I20260923 18:00:06.461426 16566 leader_election.cc:302] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e710cfbf08645118fc7cf171f63a1f9, aaccc304fad74194b27a8158864ba8d4; no voters:
I20260923 18:00:06.461899 16976 raft_consensus.cc:2781] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:06.462031 16976 raft_consensus.cc:482] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:06.462100 16976 raft_consensus.cc:3037] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [term 0 FOLLOWER]: Advancing to term 1
W20260923 18:00:06.526670 16950 tablet.cc:2369] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:00:06.556025 16976 raft_consensus.cc:504] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcfaf6031394fdda45138d8250ebc64" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42941 } } peers { permanent_uuid: "aaccc304fad74194b27a8158864ba8d4" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 33441 } } peers { permanent_uuid: "3e710cfbf08645118fc7cf171f63a1f9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38019 } }
I20260923 18:00:06.556782 16976 leader_election.cc:288] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [CANDIDATE]: Term 1 election: Requested vote from peers 1bcfaf6031394fdda45138d8250ebc64 (127.15.139.130:42941), aaccc304fad74194b27a8158864ba8d4 (127.15.139.131:33441)
I20260923 18:00:06.557413 16768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f814850951744882965409a75fe0bc3b" candidate_uuid: "3e710cfbf08645118fc7cf171f63a1f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcfaf6031394fdda45138d8250ebc64"
I20260923 18:00:06.557479 16903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f814850951744882965409a75fe0bc3b" candidate_uuid: "3e710cfbf08645118fc7cf171f63a1f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaccc304fad74194b27a8158864ba8d4"
I20260923 18:00:06.557618 16768 raft_consensus.cc:3037] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:06.557672 16903 raft_consensus.cc:3037] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:06.559016 16768 raft_consensus.cc:2454] T f814850951744882965409a75fe0bc3b P 1bcfaf6031394fdda45138d8250ebc64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e710cfbf08645118fc7cf171f63a1f9 in term 1.
I20260923 18:00:06.559643 16568 leader_election.cc:302] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcfaf6031394fdda45138d8250ebc64, 3e710cfbf08645118fc7cf171f63a1f9; no voters:
I20260923 18:00:06.559696 16903 raft_consensus.cc:2454] T f814850951744882965409a75fe0bc3b P aaccc304fad74194b27a8158864ba8d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e710cfbf08645118fc7cf171f63a1f9 in term 1.
I20260923 18:00:06.559966 16976 raft_consensus.cc:2781] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:06.560281 16976 raft_consensus.cc:686] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [term 1 LEADER]: Becoming Leader. State: Replica: 3e710cfbf08645118fc7cf171f63a1f9, State: Running, Role: LEADER
I20260923 18:00:06.560756 16976 consensus_queue.cc:237] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcfaf6031394fdda45138d8250ebc64" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42941 } } peers { permanent_uuid: "aaccc304fad74194b27a8158864ba8d4" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 33441 } } peers { permanent_uuid: "3e710cfbf08645118fc7cf171f63a1f9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38019 } }
I20260923 18:00:06.564978 16488 catalog_manager.cc:5495] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e710cfbf08645118fc7cf171f63a1f9 (127.15.139.129). New cstate: current_term: 1 leader_uuid: "3e710cfbf08645118fc7cf171f63a1f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcfaf6031394fdda45138d8250ebc64" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aaccc304fad74194b27a8158864ba8d4" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 33441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e710cfbf08645118fc7cf171f63a1f9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38019 } health_report { overall_health: HEALTHY } } }
Sep 23 18:00:06 dist-test-slave-vr57 krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.129@KRBTEST.COM
I20260923 18:00:06.585142 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.574698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34168 (local address 127.15.139.129:38019)
0923 18:00:06.574847 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:06.574852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:06.575089 (+ 237us) server_negotiation.cc:407] Connection header received
0923 18:00:06.575287 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:06.575289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:06.575340 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:06.575437 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:06.576282 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.576854 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.577869 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.578125 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.578255 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.581294 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:06.581309 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:06.581326 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:06.581355 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:06.583211 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:06.583758 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:06.583764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:06.583766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:06.583858 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:06.584139 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:06.584142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:06.584143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:06.584428 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:06.584635 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:06.584823 (+ 188us) server_negotiation.cc:299] Negotiation successful
0923 18:00:06.584941 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Sep 23 18:00:06 dist-test-slave-vr57 krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260923 18:00:06.619791 16680 tablet.cc:2369] T f814850951744882965409a75fe0bc3b P 3e710cfbf08645118fc7cf171f63a1f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 18:00:06 dist-test-slave-vr57 krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.139.129@KRBTEST.COM
I20260923 18:00:06.653476 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.642969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.15.139.129:38019)
0923 18:00:06.643142 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:06.643146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:06.643425 (+ 279us) server_negotiation.cc:407] Connection header received
0923 18:00:06.643698 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:06.643702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:06.643761 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:06.643863 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:06.644676 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.645272 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.646237 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.646508 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.646650 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.649701 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:06.649704 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:06.649707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:06.649738 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:06.651550 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:06.652051 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:06.652055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:06.652058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:06.652150 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:06.652461 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:06.652465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:06.652467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:06.652667 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:06.652814 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:06.653158 (+ 344us) server_negotiation.cc:299] Negotiation successful
0923 18:00:06.653291 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Sep 23 18:00:06 dist-test-slave-vr57 krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 18:00:06 dist-test-slave-vr57 krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.129@KRBTEST.COM
I20260923 18:00:06.740235 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:06.730513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34182 (local address 127.15.139.129:38019)
0923 18:00:06.730651 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:06.730655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:06.730905 (+ 250us) server_negotiation.cc:407] Connection header received
0923 18:00:06.731103 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:06.731108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:06.731160 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:06.731255 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:06.732163 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.732678 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.733553 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:06.733778 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:06.733890 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:06.736535 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:06.736539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:06.736541 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:06.736573 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:06.738305 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:06.738861 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:06.738864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:06.738865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:06.739011 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:06.739367 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:06.739371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:06.739372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:06.739621 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:06.739726 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:06.740007 (+ 281us) server_negotiation.cc:299] Negotiation successful
0923 18:00:06.740112 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260923 18:00:06.742138 16593 tablet_service.cc:2908] Rejecting scan request for tablet f814850951744882965409a75fe0bc3b: Uninitialized: clean time has not yet been initialized
I20260923 18:00:06.744302 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 16550
I20260923 18:00:06.750806 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 16685
I20260923 18:00:06.756426 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 16820
I20260923 18:00:06.761765 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 16456
2026-09-23T18:00:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (5645 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:08 dist-test-slave-vr57 krb5kdc[16993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:08 dist-test-slave-vr57 krb5kdc[16993](info): set up 2 sockets
Sep 23 18:00:08 dist-test-slave-vr57 krb5kdc[16993](info): commencing operation
krb5kdc: starting...
W20260923 18:00:09.024195 15918 mini_kdc.cc:121] Time spent starting KDC: real 2.239s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:09 dist-test-slave-vr57 krb5kdc[16993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:00:09.152498 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:38651
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:38651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:09.262918 17006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:09.263228 17006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:09.263303 17006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:09.267308 17006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:09.267400 17006 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 18:00:09.267432 17006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:09.267475 17006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:09.267527 17006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:09.267560 17006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:09.272696 17006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:38651
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:38651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:09.273747 17006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:09.274762 17006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:09.283699 17012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:09.283741 17011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:09.283751 17014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:09.284027 17006 server_base.cc:1034] running on GCE node
I20260923 18:00:09.284598 17006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:09.285692 17006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:09.286863 17006 hybrid_clock.cc:648] HybridClock initialized: now 1790186409286852 us; error 25 us; skew 500 ppm
Sep 23 18:00:09 dist-test-slave-vr57 krb5kdc[16993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186409, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:09.297240 17006 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:00:09.298282 17006 webserver.cc:460] Webserver started at http://127.15.139.190:35921/ using document root <none> and password file <none>
I20260923 18:00:09.298866 17006 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:09.298929 17006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:09.299173 17006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:09.301419 17006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "057c2faa66a74c6d9f6b878aba4a73a9"
format_stamp: "Formatted at 2026-09-23 18:00:09 on dist-test-slave-vr57"
server_key: "9b2d89152ec7868e1291082b52deeecf"
server_key_iv: "cfdc0e061170479780fdc3d7df7c845c"
server_key_version: "encryptionkey@0"
I20260923 18:00:09.302018 17006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "057c2faa66a74c6d9f6b878aba4a73a9"
format_stamp: "Formatted at 2026-09-23 18:00:09 on dist-test-slave-vr57"
server_key: "9b2d89152ec7868e1291082b52deeecf"
server_key_iv: "cfdc0e061170479780fdc3d7df7c845c"
server_key_version: "encryptionkey@0"
I20260923 18:00:09.305943 17006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:09.308585 17021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:09.310025 17006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:09.310184 17006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "057c2faa66a74c6d9f6b878aba4a73a9"
format_stamp: "Formatted at 2026-09-23 18:00:09 on dist-test-slave-vr57"
server_key: "9b2d89152ec7868e1291082b52deeecf"
server_key_iv: "cfdc0e061170479780fdc3d7df7c845c"
server_key_version: "encryptionkey@0"
I20260923 18:00:09.310304 17006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:09.325570 17006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:09.328881 17006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:09.329044 17006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:09.337366 17006 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:38651
I20260923 18:00:09.337380 17083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:38651 every 8 connection(s)
I20260923 18:00:09.338399 17006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:09.338675 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 17006
I20260923 18:00:09.338789 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:09.339088 15918 external_mini_cluster.cc:1428] Setting key b107a33f04edaca438bb220178f4c4e5
I20260923 18:00:09.342011 17084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 23 18:00:09 dist-test-slave-vr57 krb5kdc[16993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:09.348232 17084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9: Bootstrap starting.
I20260923 18:00:09.350783 17084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:09.352025 17084 log.cc:824] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:09.353318 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:09.340632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38870 (local address 127.15.139.190:38651)
0923 18:00:09.341105 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:09.341120 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:09.341155 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:00:09.341779 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:09.341795 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:09.342157 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:09.342503 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:09.343300 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.344343 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.345203 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.345668 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.345845 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:09.348443 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:09.348448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:09.348462 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:09.348491 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:09.350251 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:09.350723 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:09.350728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:09.350732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:09.350850 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:09.351144 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:09.351149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:09.351150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:09.351615 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:09.351809 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:09.352121 (+ 312us) server_negotiation.cc:299] Negotiation successful
0923 18:00:09.352458 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":171,"threads_started":1}
I20260923 18:00:09.354318 17084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9: No bootstrap required, opened a new log
I20260923 18:00:09.357647 17084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057c2faa66a74c6d9f6b878aba4a73a9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 38651 } }
I20260923 18:00:09.357872 17084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:09.357914 17084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 057c2faa66a74c6d9f6b878aba4a73a9, State: Initialized, Role: FOLLOWER
I20260923 18:00:09.358291 17084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057c2faa66a74c6d9f6b878aba4a73a9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 38651 } }
I20260923 18:00:09.358412 17084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:09.358461 17084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:09.358533 17084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:09.359426 17084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057c2faa66a74c6d9f6b878aba4a73a9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 38651 } }
I20260923 18:00:09.359798 17084 leader_election.cc:302] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 057c2faa66a74c6d9f6b878aba4a73a9; no voters:
I20260923 18:00:09.360198 17084 leader_election.cc:288] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:09.360313 17089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:09.360572 17089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [term 1 LEADER]: Becoming Leader. State: Replica: 057c2faa66a74c6d9f6b878aba4a73a9, State: Running, Role: LEADER
I20260923 18:00:09.361042 17089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057c2faa66a74c6d9f6b878aba4a73a9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 38651 } }
I20260923 18:00:09.361506 17084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:09.362862 17091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 057c2faa66a74c6d9f6b878aba4a73a9. Latest consensus state: current_term: 1 leader_uuid: "057c2faa66a74c6d9f6b878aba4a73a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057c2faa66a74c6d9f6b878aba4a73a9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 38651 } } }
I20260923 18:00:09.362990 17091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:09.362931 17090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "057c2faa66a74c6d9f6b878aba4a73a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057c2faa66a74c6d9f6b878aba4a73a9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 38651 } } }
I20260923 18:00:09.363066 17090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:09.363427 17098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:09.366734 17098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:09.372766 17098 catalog_manager.cc:1347] Generated new cluster ID: 206e866e19c94786b2f894549526e11c
I20260923 18:00:09.372841 17098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:09.384702 17098 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:09.385762 17098 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:09.397723 17098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9: Generated new TSK 0
I20260923 18:00:09.398548 17098 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:00:09.438849 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:38651
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:09.556354 17110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:09.556663 17110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:09.556736 17110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:09.560845 17110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:09.560931 17110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:09.561013 17110 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 18:00:09.561091 17110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:09.566568 17110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.139.190:38651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:09.567737 17110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:09.568818 17110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:09.578722 17115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:09.578866 17110 server_base.cc:1034] running on GCE node
W20260923 18:00:09.578711 17116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:09.578804 17118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:09.579427 17110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:09.580160 17110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:09.581350 17110 hybrid_clock.cc:648] HybridClock initialized: now 1790186409581317 us; error 45 us; skew 500 ppm
Sep 23 18:00:09 dist-test-slave-vr57 krb5kdc[16993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186409, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:09.584285 17110 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:00:09.585286 17110 webserver.cc:460] Webserver started at http://127.15.139.129:44291/ using document root <none> and password file <none>
I20260923 18:00:09.585855 17110 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:09.585944 17110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:09.586174 17110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:09.588290 17110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "198d76df7d5040a9ae084699952e4356"
format_stamp: "Formatted at 2026-09-23 18:00:09 on dist-test-slave-vr57"
server_key: "66bc16b54e6d36347d43a3a690b14892"
server_key_iv: "067ecb42a53d4dd147e28a0192005550"
server_key_version: "encryptionkey@0"
I20260923 18:00:09.588858 17110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "198d76df7d5040a9ae084699952e4356"
format_stamp: "Formatted at 2026-09-23 18:00:09 on dist-test-slave-vr57"
server_key: "66bc16b54e6d36347d43a3a690b14892"
server_key_iv: "067ecb42a53d4dd147e28a0192005550"
server_key_version: "encryptionkey@0"
I20260923 18:00:09.592597 17110 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260923 18:00:09.595221 17125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:09.596617 17110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:09.596760 17110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "198d76df7d5040a9ae084699952e4356"
format_stamp: "Formatted at 2026-09-23 18:00:09 on dist-test-slave-vr57"
server_key: "66bc16b54e6d36347d43a3a690b14892"
server_key_iv: "067ecb42a53d4dd147e28a0192005550"
server_key_version: "encryptionkey@0"
I20260923 18:00:09.596879 17110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:09.612233 17110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:09.615957 17110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:09.616171 17110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:09.617687 17110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:09.617762 17110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:09.617838 17110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:09.617882 17110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 18:00:09 dist-test-slave-vr57 krb5kdc[16993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186409, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:09.629647 17110 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:38383
I20260923 18:00:09.629734 17240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:38383 every 8 connection(s)
I20260923 18:00:09.630817 17110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:09.632196 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:09.618552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:39301 (local address 127.15.139.190:38651)
0923 18:00:09.618688 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:09.618692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:09.619831 (+ 1139us) server_negotiation.cc:407] Connection header received
0923 18:00:09.620925 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:09.620932 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:09.621018 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:09.621123 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:09.623045 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.623807 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.624993 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.625298 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.625411 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:09.628251 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:09.628255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:09.628258 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:09.628293 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:09.630044 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:09.630533 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:09.630537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:09.630539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:09.630635 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:09.630967 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:09.630970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:09.630972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:09.631175 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:09.631289 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:09.631894 (+ 605us) server_negotiation.cc:299] Negotiation successful
0923 18:00:09.632029 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:00:09.632977 17134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:09.619111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:38651 (local address 127.15.139.129:39301)
0923 18:00:09.619681 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:09.619724 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:09.620642 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:09.621468 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:09.621480 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:09.622014 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:09.622837 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:09.622863 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.623943 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.623947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:09.624861 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:09.624870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.625461 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.625465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:09.625676 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:09.626301 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:09.626316 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:09.628080 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:09.630157 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:09.630161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:09.630173 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:09.630424 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:09.630730 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:09.630732 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:09.630733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:09.630861 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:09.631287 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:09.631294 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:09.631633 (+ 339us) client_negotiation.cc:769] Sending connection context
0923 18:00:09.631850 (+ 217us) client_negotiation.cc:240] Negotiation successful
0923 18:00:09.632182 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":152,"threads_started":1}
I20260923 18:00:09.634279 17241 heartbeater.cc:347] Connected to a master server at 127.15.139.190:38651
I20260923 18:00:09.634411 17241 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:09.634943 17241 heartbeater.cc:511] Master 127.15.139.190:38651 requested a full tablet report, sending...
I20260923 18:00:09.635732 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 17110
I20260923 18:00:09.635854 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:09.636147 15918 external_mini_cluster.cc:1428] Setting key 4c963c9f64471c1e5769898cba9b62b8
I20260923 18:00:09.636693 17036 ts_manager.cc:194] Registered new tserver with Master: 198d76df7d5040a9ae084699952e4356 (127.15.139.129:38383)
I20260923 18:00:09.638134 17036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:39301
I20260923 18:00:09.645313 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:09.636588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:38651 (local address 127.15.139.129:39207)
0923 18:00:09.636903 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:09.636935 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:09.637035 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:09.637275 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:09.637278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:09.637526 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:09.637660 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:09.637668 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.638638 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.638645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:09.639677 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:09.639691 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.640487 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.640490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:09.640599 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:09.641123 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:09.641134 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:09.641491 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:09.643612 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:09.643616 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:09.643618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:09.643836 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:09.644247 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:09.644249 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:09.644251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:09.644331 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:09.644798 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:09.644800 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:09.644946 (+ 146us) client_negotiation.cc:769] Sending connection context
0923 18:00:09.645037 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 18:00:09.645162 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":91,"threads_started":1}
I20260923 18:00:09.645431 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:09.636676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:39207 (local address 127.15.139.190:38651)
0923 18:00:09.636796 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:09.636801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:09.637020 (+ 219us) server_negotiation.cc:407] Connection header received
0923 18:00:09.637120 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:09.637123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:09.637163 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:09.637245 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:09.637793 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.638494 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.639823 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.640234 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.640328 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:09.641642 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:09.641649 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:09.641655 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:09.641687 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:09.643420 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:09.644027 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:09.644034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:09.644037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:09.644143 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:09.644430 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:09.644435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:09.644440 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:09.644690 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:09.644798 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:09.645124 (+ 326us) server_negotiation.cc:299] Negotiation successful
0923 18:00:09.645255 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:00:09.666388 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:38651
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:09.778337 17250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:09.778589 17250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:09.778637 17250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:09.782850 17250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:09.782913 17250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:09.782956 17250 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 18:00:09.783059 17250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:09.788533 17250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.139.190:38651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:09.789558 17250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:09.790587 17250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:09.800561 17256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:09.800619 17255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:09.800565 17258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:09.800767 17250 server_base.cc:1034] running on GCE node
I20260923 18:00:09.801268 17250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:09.801890 17250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:09.803124 17250 hybrid_clock.cc:648] HybridClock initialized: now 1790186409803114 us; error 44 us; skew 500 ppm
Sep 23 18:00:09 dist-test-slave-vr57 krb5kdc[16993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186409, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:09.805959 17250 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:00:09.806824 17250 webserver.cc:460] Webserver started at http://127.15.139.130:40463/ using document root <none> and password file <none>
I20260923 18:00:09.807389 17250 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:09.807510 17250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:09.807853 17250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:09.809939 17250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "15d5429fbf68488c84799bc2496d5b38"
format_stamp: "Formatted at 2026-09-23 18:00:09 on dist-test-slave-vr57"
server_key: "10e9fddcbfba855402b4d1ac0fce8490"
server_key_iv: "cd7add6387deaabb73897bdec6d869bc"
server_key_version: "encryptionkey@0"
I20260923 18:00:09.810503 17250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "15d5429fbf68488c84799bc2496d5b38"
format_stamp: "Formatted at 2026-09-23 18:00:09 on dist-test-slave-vr57"
server_key: "10e9fddcbfba855402b4d1ac0fce8490"
server_key_iv: "cd7add6387deaabb73897bdec6d869bc"
server_key_version: "encryptionkey@0"
I20260923 18:00:09.814380 17250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260923 18:00:09.816915 17265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:09.818198 17250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:09.818382 17250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "15d5429fbf68488c84799bc2496d5b38"
format_stamp: "Formatted at 2026-09-23 18:00:09 on dist-test-slave-vr57"
server_key: "10e9fddcbfba855402b4d1ac0fce8490"
server_key_iv: "cd7add6387deaabb73897bdec6d869bc"
server_key_version: "encryptionkey@0"
I20260923 18:00:09.818514 17250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:09.840093 17250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:09.843091 17250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:09.843320 17250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:09.844918 17250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:09.845001 17250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:09.845074 17250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:09.845129 17250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 18:00:09 dist-test-slave-vr57 krb5kdc[16993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186409, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:09.857210 17250 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:44535
I20260923 18:00:09.857333 17380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:44535 every 8 connection(s)
I20260923 18:00:09.858459 17250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:09.860287 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:09.845791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:33609 (local address 127.15.139.190:38651)
0923 18:00:09.846079 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:09.846087 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:09.846839 (+ 752us) server_negotiation.cc:407] Connection header received
0923 18:00:09.847922 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:09.847926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:09.847984 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:09.848081 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:09.849848 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.850749 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.852404 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.852764 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.852904 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:09.856190 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:09.856194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:09.856196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:09.856233 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:09.857925 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:09.858504 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:09.858509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:09.858511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:09.858629 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:09.859009 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:09.859014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:09.859017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:09.859261 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:09.859364 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:09.860054 (+ 690us) server_negotiation.cc:299] Negotiation successful
0923 18:00:09.860168 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260923 18:00:09.861020 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:09.846197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:38651 (local address 127.15.139.130:33609)
0923 18:00:09.846658 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:09.846708 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:09.847703 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:09.848410 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:09.848421 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:09.848964 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:09.849663 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:09.849685 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.850929 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.850933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:09.852272 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:09.852282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.852948 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.852955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:09.853211 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:09.853942 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:09.853967 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:09.856003 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:09.858052 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:09.858057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:09.858068 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:09.858367 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:09.858725 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:09.858727 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:09.858729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:09.858882 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:09.859420 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:09.859425 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:09.859731 (+ 306us) client_negotiation.cc:769] Sending connection context
0923 18:00:09.859884 (+ 153us) client_negotiation.cc:240] Negotiation successful
0923 18:00:09.860134 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":132,"threads_started":1}
I20260923 18:00:09.862272 17381 heartbeater.cc:347] Connected to a master server at 127.15.139.190:38651
I20260923 18:00:09.862402 17381 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:09.863049 17381 heartbeater.cc:511] Master 127.15.139.190:38651 requested a full tablet report, sending...
I20260923 18:00:09.863395 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 17250
I20260923 18:00:09.863520 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:09.863837 15918 external_mini_cluster.cc:1428] Setting key 3ac3d7f69590af7e289efb8625e4aeba
I20260923 18:00:09.865231 17036 ts_manager.cc:194] Registered new tserver with Master: 15d5429fbf68488c84799bc2496d5b38 (127.15.139.130:44535)
I20260923 18:00:09.865983 17036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:33609
I20260923 18:00:09.872697 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:09.864970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:47973 (local address 127.15.139.190:38651)
0923 18:00:09.865087 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:09.865091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:09.865105 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:00:09.865139 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:09.865141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:09.865180 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:09.865257 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:09.865837 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.866529 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.867725 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.868088 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.868203 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:09.869505 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:09.869509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:09.869511 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:09.869531 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:09.870978 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:09.871485 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:09.871488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:09.871489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:09.871614 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:09.871918 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:09.871919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:09.871921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:09.872108 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:09.872183 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:09.872480 (+ 297us) server_negotiation.cc:299] Negotiation successful
0923 18:00:09.872587 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 18:00:09.872833 17387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:09.864659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:38651 (local address 127.15.139.130:47973)
0923 18:00:09.864930 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:09.864945 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:09.865040 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:09.865374 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:09.865376 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:09.865585 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:09.865713 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:09.865721 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.866726 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.866730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:09.867579 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:09.867589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:09.868234 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:09.868240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:09.868362 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:09.868865 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:09.868885 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:09.869363 (+ 478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:09.871096 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:09.871102 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:09.871107 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:09.871389 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:09.871711 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:09.871717 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:09.871722 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:09.871823 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:09.872213 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:09.872216 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:09.872318 (+ 102us) client_negotiation.cc:769] Sending connection context
0923 18:00:09.872435 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 18:00:09.872592 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":100,"threads_started":1}
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:00:09.892596 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:38651
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:10.007071 17390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:10.007401 17390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:10.007476 17390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:10.012400 17390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:10.012496 17390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:10.012568 17390 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 18:00:10.012638 17390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:10.018329 17390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.139.190:38651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:10.019558 17390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:10.020622 17390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:10.030581 17396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:10.030555 17395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:10.030560 17398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:10.030788 17390 server_base.cc:1034] running on GCE node
I20260923 18:00:10.031355 17390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:10.032117 17390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:10.033326 17390 hybrid_clock.cc:648] HybridClock initialized: now 1790186410033274 us; error 69 us; skew 500 ppm
Sep 23 18:00:10 dist-test-slave-vr57 krb5kdc[16993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:10.036207 17390 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:00:10.037009 17390 webserver.cc:460] Webserver started at http://127.15.139.131:41343/ using document root <none> and password file <none>
I20260923 18:00:10.037518 17390 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:10.037575 17390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:10.037731 17390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:10.039795 17390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0"
format_stamp: "Formatted at 2026-09-23 18:00:10 on dist-test-slave-vr57"
server_key: "347157facc6ca5ec64aeda431da443d2"
server_key_iv: "6bb2b91ae8226975f566029591af001f"
server_key_version: "encryptionkey@0"
I20260923 18:00:10.040295 17390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0"
format_stamp: "Formatted at 2026-09-23 18:00:10 on dist-test-slave-vr57"
server_key: "347157facc6ca5ec64aeda431da443d2"
server_key_iv: "6bb2b91ae8226975f566029591af001f"
server_key_version: "encryptionkey@0"
I20260923 18:00:10.043849 17390 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:00:10.046195 17405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:10.047394 17390 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:10.047484 17390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0"
format_stamp: "Formatted at 2026-09-23 18:00:10 on dist-test-slave-vr57"
server_key: "347157facc6ca5ec64aeda431da443d2"
server_key_iv: "6bb2b91ae8226975f566029591af001f"
server_key_version: "encryptionkey@0"
I20260923 18:00:10.047628 17390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:10.069885 17390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:10.072970 17390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:10.073140 17390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:10.074489 17390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:10.074568 17390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:10.074638 17390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:10.074683 17390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 18:00:10 dist-test-slave-vr57 krb5kdc[16993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:10.085927 17390 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:46185
I20260923 18:00:10.086083 17520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:46185 every 8 connection(s)
I20260923 18:00:10.087126 17390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:10.089231 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.075635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:32835 (local address 127.15.139.190:38651)
0923 18:00:10.075764 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.075768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.076855 (+ 1087us) server_negotiation.cc:407] Connection header received
0923 18:00:10.078132 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.078137 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.078206 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.078308 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:10.079802 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.080557 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.081842 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.082084 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.082201 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.085078 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:10.085082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:10.085085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:10.085113 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:10.087013 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:10.087672 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:10.087679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:10.087682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:10.087787 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:10.088216 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:10.088219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:10.088221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:10.088404 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:10.088481 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.088986 (+ 505us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.089113 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:00:10.089526 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 17390
I20260923 18:00:10.089658 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:00:10.089958 15918 external_mini_cluster.cc:1428] Setting key 1e5b7dd0e6468fc64e84f069378e69f8
I20260923 18:00:10.091352 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.076206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:38651 (local address 127.15.139.131:32835)
0923 18:00:10.076706 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.076748 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.077795 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.078556 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.078565 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.078929 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:10.079655 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.079671 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.080677 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.080680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.081716 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.081725 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.082261 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.082266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.082445 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.083054 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:10.083070 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:10.084886 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:10.087138 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:10.087145 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:10.087156 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:10.087558 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:10.087885 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:10.087891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:10.087896 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:10.088102 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:10.088502 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:10.088507 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:10.088768 (+ 261us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.088926 (+ 158us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.089229 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":158,"threads_started":1}
I20260923 18:00:10.092700 17521 heartbeater.cc:347] Connected to a master server at 127.15.139.190:38651
I20260923 18:00:10.092859 17521 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:10.093365 17521 heartbeater.cc:511] Master 127.15.139.190:38651 requested a full tablet report, sending...
I20260923 18:00:10.094450 17038 ts_manager.cc:194] Registered new tserver with Master: d1bd1a157a114ae1b2e8b84cfe4ef0e0 (127.15.139.131:46185)
I20260923 18:00:10.095301 17038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:32835
I20260923 18:00:10.102015 17527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.094804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:38651 (local address 127.15.139.131:39267)
0923 18:00:10.095195 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.095206 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.095315 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.095766 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.095768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.096021 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:10.096136 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.096144 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.096886 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.096890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.097628 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.097637 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.098054 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.098057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.098121 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.098579 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:10.098590 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:10.098891 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:10.100463 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:10.100468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:10.100474 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:10.100662 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:10.100899 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:10.100901 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:10.100902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:10.100992 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:10.101402 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:10.101405 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:10.101556 (+ 151us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.101677 (+ 121us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.101822 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":95,"threads_started":1}
I20260923 18:00:10.102033 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.094889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:39267 (local address 127.15.139.190:38651)
0923 18:00:10.095012 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.095016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.095296 (+ 280us) server_negotiation.cc:407] Connection header received
0923 18:00:10.095454 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.095457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.095554 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.095615 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:10.096255 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.096744 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.097728 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.097927 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.098010 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.099009 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:10.099012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:10.099014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:10.099034 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:10.100375 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:10.100746 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:10.100749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:10.100750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:10.100820 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:10.101099 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:10.101121 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:10.101126 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:10.101300 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:10.101414 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.101733 (+ 319us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.101838 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:00:10.104328 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 18:00:10 dist-test-slave-vr57 krb5kdc[16993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186410, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 18:00:10 dist-test-slave-vr57 krb5kdc[16993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186410, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:10.130959 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.122203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.15.139.190:38651)
0923 18:00:10.122334 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.122338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.122351 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:00:10.122396 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.122398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.122440 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.122514 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:10.123422 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.124001 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.124921 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.125160 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.125242 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.127940 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:10.127944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:10.127946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:10.127973 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:10.129319 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:10.129741 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:10.129745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:10.129746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:10.129831 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:10.130084 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:10.130086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:10.130087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:10.130273 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:10.130380 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.130613 (+ 233us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.130716 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 18:00:10.134231 17038 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:00:10.137504 17038 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:10.152822 17543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.147557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37366 (local address 127.15.139.130:44535)
0923 18:00:10.147976 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.147989 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.148012 (+ 23us) server_negotiation.cc:407] Connection header received
0923 18:00:10.148164 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.148168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.148309 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.148751 (+ 442us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:10.148903 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.149609 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.151344 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.152091 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.152256 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.152402 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.152526 (+ 124us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.152631 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":111,"threads_started":1}
I20260923 18:00:10.152809 17541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.147108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:44535 (local address 127.0.0.1:37366)
0923 18:00:10.147655 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.147670 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.147749 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.148462 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.148465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.148485 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:10.148602 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.148609 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.149729 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.149735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.151220 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.151229 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.152220 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.152225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.152342 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.152378 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.152496 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.152625 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":127,"thread_start_us":91,"threads_started":1}
I20260923 18:00:10.152848 17539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.146501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:46185 (local address 127.0.0.1:35762)
0923 18:00:10.147248 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.147262 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.147395 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.148426 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.148432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.148458 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:10.148585 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.148593 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.149985 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.149989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.151160 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.151170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.152367 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.152371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.152465 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.152495 (+ 30us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.152552 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.152626 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":3,"spinlock_wait_cycles":15104,"thread_start_us":128,"threads_started":1}
I20260923 18:00:10.153772 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.147078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35762 (local address 127.15.139.131:46185)
0923 18:00:10.147988 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.148000 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.148017 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:00:10.148088 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.148095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.148273 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.148446 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:10.148734 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.149840 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.151300 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.152233 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.153296 (+ 1063us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.153426 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.153490 (+ 64us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.153582 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":129,"threads_started":1}
I20260923 18:00:10.154003 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.146533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.15.139.129:38383)
0923 18:00:10.146845 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.146856 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.147264 (+ 408us) server_negotiation.cc:407] Connection header received
0923 18:00:10.147634 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.147642 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.147863 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.147997 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:10.148519 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.149705 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.152242 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.153059 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.153185 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.153327 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.153745 (+ 418us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.153857 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":110,"threads_started":1}
I20260923 18:00:10.154417 17538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.146396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:38383 (local address 127.0.0.1:58910)
0923 18:00:10.147104 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.147160 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.147396 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.148205 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.148209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.148229 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:10.148345 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.148351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.149797 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.149800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.152082 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.152094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.153476 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.153480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.153573 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.153597 (+ 24us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.154213 (+ 616us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.154309 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":94,"spinlock_wait_cycles":768,"thread_start_us":66,"threads_started":1}
I20260923 18:00:10.156157 17315 tablet_service.cc:1467] Processing CreateTablet for tablet 2f0f5846215c436b80f9022f839e490b (DEFAULT_TABLE table=test-table [id=8e754e29cb47433099e56efefef9d1cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:10.156539 17455 tablet_service.cc:1467] Processing CreateTablet for tablet 2f0f5846215c436b80f9022f839e490b (DEFAULT_TABLE table=test-table [id=8e754e29cb47433099e56efefef9d1cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:10.157317 17315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f0f5846215c436b80f9022f839e490b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:10.157086 17175 tablet_service.cc:1467] Processing CreateTablet for tablet 2f0f5846215c436b80f9022f839e490b (DEFAULT_TABLE table=test-table [id=8e754e29cb47433099e56efefef9d1cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:10.157575 17455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f0f5846215c436b80f9022f839e490b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:10.158039 17175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f0f5846215c436b80f9022f839e490b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:10.164690 17544 tablet_bootstrap.cc:492] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Bootstrap starting.
I20260923 18:00:10.165740 17546 tablet_bootstrap.cc:492] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38: Bootstrap starting.
I20260923 18:00:10.167068 17544 tablet_bootstrap.cc:654] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:10.167593 17546 tablet_bootstrap.cc:654] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:10.167873 17545 tablet_bootstrap.cc:492] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356: Bootstrap starting.
I20260923 18:00:10.168027 17544 log.cc:824] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:10.168412 17546 log.cc:824] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:10.170044 17545 tablet_bootstrap.cc:654] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:10.170666 17544 tablet_bootstrap.cc:492] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0: No bootstrap required, opened a new log
I20260923 18:00:10.170871 17544 ts_tablet_manager.cc:1397] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260923 18:00:10.170848 17545 log.cc:824] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:10.171476 17546 tablet_bootstrap.cc:492] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38: No bootstrap required, opened a new log
I20260923 18:00:10.171717 17546 ts_tablet_manager.cc:1397] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260923 18:00:10.172739 17545 tablet_bootstrap.cc:492] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356: No bootstrap required, opened a new log
I20260923 18:00:10.172920 17545 ts_tablet_manager.cc:1397] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260923 18:00:10.174706 17546 raft_consensus.cc:348] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } }
I20260923 18:00:10.174961 17546 raft_consensus.cc:374] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:10.175015 17546 raft_consensus.cc:729] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15d5429fbf68488c84799bc2496d5b38, State: Initialized, Role: FOLLOWER
I20260923 18:00:10.174963 17544 raft_consensus.cc:348] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } }
I20260923 18:00:10.175251 17544 raft_consensus.cc:374] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:10.175315 17544 raft_consensus.cc:729] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bd1a157a114ae1b2e8b84cfe4ef0e0, State: Initialized, Role: FOLLOWER
I20260923 18:00:10.175477 17546 consensus_queue.cc:260] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } }
I20260923 18:00:10.175789 17544 consensus_queue.cc:260] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } }
I20260923 18:00:10.176823 17544 ts_tablet_manager.cc:1428] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260923 18:00:10.176812 17381 heartbeater.cc:503] Master 127.15.139.190:38651 was elected leader, sending a full tablet report...
I20260923 18:00:10.176822 17546 ts_tablet_manager.cc:1428] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260923 18:00:10.177278 17521 heartbeater.cc:503] Master 127.15.139.190:38651 was elected leader, sending a full tablet report...
I20260923 18:00:10.177413 17545 raft_consensus.cc:348] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } }
I20260923 18:00:10.177666 17545 raft_consensus.cc:374] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:10.177721 17545 raft_consensus.cc:729] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 198d76df7d5040a9ae084699952e4356, State: Initialized, Role: FOLLOWER
I20260923 18:00:10.178153 17545 consensus_queue.cc:260] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } }
I20260923 18:00:10.178751 17241 heartbeater.cc:503] Master 127.15.139.190:38651 was elected leader, sending a full tablet report...
I20260923 18:00:10.179906 17545 ts_tablet_manager.cc:1428] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260923 18:00:10.270358 17550 raft_consensus.cc:482] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:10.270584 17550 raft_consensus.cc:504] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } }
I20260923 18:00:10.271787 17550 leader_election.cc:288] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 198d76df7d5040a9ae084699952e4356 (127.15.139.129:38383), 15d5429fbf68488c84799bc2496d5b38 (127.15.139.130:44535)
I20260923 18:00:10.275350 17543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.272170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:58455 (local address 127.15.139.130:44535)
0923 18:00:10.272305 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.272310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.272328 (+ 18us) server_negotiation.cc:407] Connection header received
0923 18:00:10.272388 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.272391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.272440 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.272552 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:10.272825 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.273348 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.274384 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.274798 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.274929 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.274982 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.275165 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.275239 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 18:00:10.275350 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.272063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:44535 (local address 127.15.139.131:58455)
0923 18:00:10.272178 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.272196 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.272295 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.272563 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.272566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.272585 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:10.272706 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.272712 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.273463 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.273469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.274253 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.274262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.274921 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.274924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.275037 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.275061 (+ 24us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.275149 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.275215 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 18:00:10.276037 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0f5846215c436b80f9022f839e490b" candidate_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15d5429fbf68488c84799bc2496d5b38" is_pre_election: true
I20260923 18:00:10.276320 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.272210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:57095 (local address 127.15.139.129:38383)
0923 18:00:10.272329 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.272334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.272692 (+ 358us) server_negotiation.cc:407] Connection header received
0923 18:00:10.272837 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.272845 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.272889 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.272981 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:10.273250 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.273730 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.275160 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.275752 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.275921 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.275973 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.276146 (+ 173us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.276212 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 18:00:10.276340 17553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.272074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:38383 (local address 127.15.139.131:57095)
0923 18:00:10.272586 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.272598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.272715 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.272992 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.272994 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.273009 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:10.273136 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.273141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.273851 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.273854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.275028 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.275037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.275900 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.275903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.276011 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.276031 (+ 20us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.276121 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.276189 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":154,"thread_start_us":121,"threads_started":1}
I20260923 18:00:10.276353 17335 raft_consensus.cc:2454] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1bd1a157a114ae1b2e8b84cfe4ef0e0 in term 0.
I20260923 18:00:10.276937 17407 leader_election.cc:302] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15d5429fbf68488c84799bc2496d5b38, d1bd1a157a114ae1b2e8b84cfe4ef0e0; no voters:
I20260923 18:00:10.277009 17195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0f5846215c436b80f9022f839e490b" candidate_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "198d76df7d5040a9ae084699952e4356" is_pre_election: true
I20260923 18:00:10.277261 17550 raft_consensus.cc:2781] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:10.277268 17195 raft_consensus.cc:2454] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1bd1a157a114ae1b2e8b84cfe4ef0e0 in term 0.
I20260923 18:00:10.277336 17550 raft_consensus.cc:482] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:10.277367 17550 raft_consensus.cc:3037] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:10.278533 17550 raft_consensus.cc:504] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } }
I20260923 18:00:10.278891 17550 leader_election.cc:288] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [CANDIDATE]: Term 1 election: Requested vote from peers 198d76df7d5040a9ae084699952e4356 (127.15.139.129:38383), 15d5429fbf68488c84799bc2496d5b38 (127.15.139.130:44535)
I20260923 18:00:10.279264 17195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0f5846215c436b80f9022f839e490b" candidate_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "198d76df7d5040a9ae084699952e4356"
I20260923 18:00:10.279264 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0f5846215c436b80f9022f839e490b" candidate_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15d5429fbf68488c84799bc2496d5b38"
I20260923 18:00:10.279387 17195 raft_consensus.cc:3037] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:10.279410 17335 raft_consensus.cc:3037] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:10.280851 17195 raft_consensus.cc:2454] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1bd1a157a114ae1b2e8b84cfe4ef0e0 in term 1.
I20260923 18:00:10.281149 17406 leader_election.cc:302] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 198d76df7d5040a9ae084699952e4356, d1bd1a157a114ae1b2e8b84cfe4ef0e0; no voters:
I20260923 18:00:10.281343 17550 raft_consensus.cc:2781] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:10.281387 17335 raft_consensus.cc:2454] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1bd1a157a114ae1b2e8b84cfe4ef0e0 in term 1.
I20260923 18:00:10.281577 17550 raft_consensus.cc:686] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 1 LEADER]: Becoming Leader. State: Replica: d1bd1a157a114ae1b2e8b84cfe4ef0e0, State: Running, Role: LEADER
I20260923 18:00:10.281944 17550 consensus_queue.cc:237] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } }
I20260923 18:00:10.285099 17035 catalog_manager.cc:5495] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1bd1a157a114ae1b2e8b84cfe4ef0e0 (127.15.139.131). New cstate: current_term: 1 leader_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } health_report { overall_health: UNKNOWN } } }
I20260923 18:00:10.311673 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.307863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.15.139.131:46185)
0923 18:00:10.307996 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.308001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.308018 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:00:10.308209 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.308211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.308266 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.308354 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:10.308622 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.309113 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.310052 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.310274 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.310367 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.310723 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:10.311192 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:10.311290 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.311420 (+ 130us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.311476 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:00:10.318358 17195 raft_consensus.cc:1261] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356 [term 1 FOLLOWER]: Refusing update from remote peer d1bd1a157a114ae1b2e8b84cfe4ef0e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:10.318360 17335 raft_consensus.cc:1261] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38 [term 1 FOLLOWER]: Refusing update from remote peer d1bd1a157a114ae1b2e8b84cfe4ef0e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:10.319126 17554 consensus_queue.cc:1035] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:10.319351 17550 consensus_queue.cc:1035] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:10.329531 17559 mvcc.cc:205] Tried to move back new op lower bound from 7332603536656113664 to 7332603536517419008. Current Snapshot: MvccSnapshot[applied={T|T < 7332603536656113664}]
I20260923 18:00:10.341861 17038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:10.342046 17038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:10.344731 17038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9: Sending DeleteTablet for 3 replicas of tablet 2f0f5846215c436b80f9022f839e490b
I20260923 18:00:10.345634 17175 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f0f5846215c436b80f9022f839e490b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:10 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:58910
I20260923 18:00:10.345636 17455 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f0f5846215c436b80f9022f839e490b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:10 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:35762
I20260923 18:00:10.346177 17568 tablet_replica.cc:330] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356: stopping tablet replica
I20260923 18:00:10.346287 17315 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f0f5846215c436b80f9022f839e490b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:10 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:37366
I20260923 18:00:10.346676 17568 raft_consensus.cc:2229] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:10.347090 17568 raft_consensus.cc:2258] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:10.347728 17567 tablet_replica.cc:330] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0: stopping tablet replica
I20260923 18:00:10.347669 17038 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 18:00:10.348018 17567 raft_consensus.cc:2229] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 1 LEADER]: Raft consensus shutting down.
W20260923 18:00:10.348091 17038 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:10.348387 17567 raft_consensus.cc:2258] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:10.349797 17567 ts_tablet_manager.cc:1905] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:10.352100 17568 ts_tablet_manager.cc:1905] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:10.353958 17315 tablet_service.cc:1467] Processing CreateTablet for tablet ef9982b88f534394bbadcf23b0487b1f (DEFAULT_TABLE table=test-table [id=92e1a133920343449b92953879ff5d5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:10.353988 17175 tablet_service.cc:1467] Processing CreateTablet for tablet ef9982b88f534394bbadcf23b0487b1f (DEFAULT_TABLE table=test-table [id=92e1a133920343449b92953879ff5d5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:10.354231 17175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef9982b88f534394bbadcf23b0487b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:10.354252 17315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef9982b88f534394bbadcf23b0487b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:10.355027 17568 ts_tablet_manager.cc:1918] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:10.355273 17568 log.cc:1196] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/2f0f5846215c436b80f9022f839e490b
I20260923 18:00:10.355697 17569 tablet_replica.cc:330] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38: stopping tablet replica
I20260923 18:00:10.355713 17568 ts_tablet_manager.cc:1939] T 2f0f5846215c436b80f9022f839e490b P 198d76df7d5040a9ae084699952e4356: Deleting consensus metadata
I20260923 18:00:10.356001 17569 raft_consensus.cc:2229] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:10.356314 17569 raft_consensus.cc:2258] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:10.356534 17546 tablet_bootstrap.cc:492] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38: Bootstrap starting.
I20260923 18:00:10.356559 17545 tablet_bootstrap.cc:492] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356: Bootstrap starting.
I20260923 18:00:10.356743 17455 tablet_service.cc:1467] Processing CreateTablet for tablet ef9982b88f534394bbadcf23b0487b1f (DEFAULT_TABLE table=test-table [id=92e1a133920343449b92953879ff5d5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:10.357010 17455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef9982b88f534394bbadcf23b0487b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:10.357424 17545 tablet_bootstrap.cc:654] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:10.357496 17546 tablet_bootstrap.cc:654] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:10.358219 17022 catalog_manager.cc:4841] TS 198d76df7d5040a9ae084699952e4356 (127.15.139.129:38383): tablet 2f0f5846215c436b80f9022f839e490b (table test-table [id=8e754e29cb47433099e56efefef9d1cc]) successfully deleted
I20260923 18:00:10.359915 17544 tablet_bootstrap.cc:492] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Bootstrap starting.
I20260923 18:00:10.360164 17569 ts_tablet_manager.cc:1905] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:10.360864 17544 tablet_bootstrap.cc:654] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:10.362475 17567 ts_tablet_manager.cc:1918] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:10.362576 17567 log.cc:1196] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/2f0f5846215c436b80f9022f839e490b
I20260923 18:00:10.362993 17569 ts_tablet_manager.cc:1918] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:10.363020 17567 ts_tablet_manager.cc:1939] T 2f0f5846215c436b80f9022f839e490b P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Deleting consensus metadata
I20260923 18:00:10.363080 17569 log.cc:1196] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/2f0f5846215c436b80f9022f839e490b
I20260923 18:00:10.363358 17569 ts_tablet_manager.cc:1939] T 2f0f5846215c436b80f9022f839e490b P 15d5429fbf68488c84799bc2496d5b38: Deleting consensus metadata
I20260923 18:00:10.364233 17023 catalog_manager.cc:4841] TS d1bd1a157a114ae1b2e8b84cfe4ef0e0 (127.15.139.131:46185): tablet 2f0f5846215c436b80f9022f839e490b (table test-table [id=8e754e29cb47433099e56efefef9d1cc]) successfully deleted
I20260923 18:00:10.364468 17023 catalog_manager.cc:4841] TS 15d5429fbf68488c84799bc2496d5b38 (127.15.139.130:44535): tablet 2f0f5846215c436b80f9022f839e490b (table test-table [id=8e754e29cb47433099e56efefef9d1cc]) successfully deleted
I20260923 18:00:10.365159 17544 tablet_bootstrap.cc:492] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0: No bootstrap required, opened a new log
I20260923 18:00:10.365235 17544 ts_tablet_manager.cc:1397] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260923 18:00:10.365473 17545 tablet_bootstrap.cc:492] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356: No bootstrap required, opened a new log
I20260923 18:00:10.365471 17546 tablet_bootstrap.cc:492] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38: No bootstrap required, opened a new log
I20260923 18:00:10.365552 17546 ts_tablet_manager.cc:1397] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20260923 18:00:10.365552 17545 ts_tablet_manager.cc:1397] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260923 18:00:10.365661 17544 raft_consensus.cc:348] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.365764 17544 raft_consensus.cc:374] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:10.365794 17544 raft_consensus.cc:729] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bd1a157a114ae1b2e8b84cfe4ef0e0, State: Initialized, Role: FOLLOWER
I20260923 18:00:10.365906 17544 consensus_queue.cc:260] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.366004 17546 raft_consensus.cc:348] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.366104 17546 raft_consensus.cc:374] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:10.366134 17546 raft_consensus.cc:729] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15d5429fbf68488c84799bc2496d5b38, State: Initialized, Role: FOLLOWER
I20260923 18:00:10.366147 17544 ts_tablet_manager.cc:1428] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:10.366250 17546 consensus_queue.cc:260] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.366334 17545 raft_consensus.cc:348] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.366429 17545 raft_consensus.cc:374] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:10.366461 17545 raft_consensus.cc:729] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 198d76df7d5040a9ae084699952e4356, State: Initialized, Role: FOLLOWER
I20260923 18:00:10.366493 17546 ts_tablet_manager.cc:1428] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260923 18:00:10.366600 17545 consensus_queue.cc:260] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.366860 17545 ts_tablet_manager.cc:1428] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20260923 18:00:10.382597 17242 tablet.cc:2369] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:00:10.425648 17552 raft_consensus.cc:482] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:10.425814 17552 raft_consensus.cc:504] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.427109 17552 leader_election.cc:288] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15d5429fbf68488c84799bc2496d5b38 (127.15.139.130:44535), d1bd1a157a114ae1b2e8b84cfe4ef0e0 (127.15.139.131:46185)
I20260923 18:00:10.431805 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.427412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:44535 (local address 127.15.139.129:52159)
0923 18:00:10.427677 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.427692 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.427804 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.428162 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.428166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.428186 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:10.428301 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.428308 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.429168 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.429172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.430500 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.430511 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.431319 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.431323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.431422 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.431450 (+ 28us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.431582 (+ 132us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.431659 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":117,"threads_started":1}
I20260923 18:00:10.431792 17543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.427466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:52159 (local address 127.15.139.130:44535)
0923 18:00:10.427628 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.427634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.427767 (+ 133us) server_negotiation.cc:407] Connection header received
0923 18:00:10.427911 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.427914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.427963 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.428046 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:10.428505 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.429058 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.430640 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.431180 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.431317 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.431376 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.431600 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.431670 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 18:00:10.432328 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.427813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:46185 (local address 127.15.139.129:46351)
0923 18:00:10.428174 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.428187 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.428284 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.429176 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.429178 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.429194 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:10.429311 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.429315 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.430018 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.430022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.430910 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.430919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.431876 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.431880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.431980 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.432003 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.432106 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.432173 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":9,"thread_start_us":94,"threads_started":1}
I20260923 18:00:10.432340 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.428062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:46351 (local address 127.15.139.131:46185)
0923 18:00:10.428177 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.428183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.428244 (+ 61us) server_negotiation.cc:407] Connection header received
0923 18:00:10.428369 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.428377 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.428416 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.428525 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:10.429426 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.429900 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.431033 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.431653 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.431767 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.431817 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.432154 (+ 337us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.432234 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 18:00:10.432511 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9982b88f534394bbadcf23b0487b1f" candidate_uuid: "198d76df7d5040a9ae084699952e4356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15d5429fbf68488c84799bc2496d5b38" is_pre_election: true
I20260923 18:00:10.432637 17335 raft_consensus.cc:2454] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 198d76df7d5040a9ae084699952e4356 in term 0.
I20260923 18:00:10.433064 17127 leader_election.cc:302] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15d5429fbf68488c84799bc2496d5b38, 198d76df7d5040a9ae084699952e4356; no voters:
I20260923 18:00:10.433274 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9982b88f534394bbadcf23b0487b1f" candidate_uuid: "198d76df7d5040a9ae084699952e4356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" is_pre_election: true
I20260923 18:00:10.433360 17552 raft_consensus.cc:2781] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:10.433429 17552 raft_consensus.cc:482] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:10.433478 17552 raft_consensus.cc:3037] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:10.433508 17475 raft_consensus.cc:2454] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 198d76df7d5040a9ae084699952e4356 in term 0.
I20260923 18:00:10.434680 17552 raft_consensus.cc:504] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.435019 17552 leader_election.cc:288] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [CANDIDATE]: Term 1 election: Requested vote from peers 15d5429fbf68488c84799bc2496d5b38 (127.15.139.130:44535), d1bd1a157a114ae1b2e8b84cfe4ef0e0 (127.15.139.131:46185)
I20260923 18:00:10.435372 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9982b88f534394bbadcf23b0487b1f" candidate_uuid: "198d76df7d5040a9ae084699952e4356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0"
I20260923 18:00:10.435407 17335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9982b88f534394bbadcf23b0487b1f" candidate_uuid: "198d76df7d5040a9ae084699952e4356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15d5429fbf68488c84799bc2496d5b38"
I20260923 18:00:10.435531 17475 raft_consensus.cc:3037] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:10.435532 17335 raft_consensus.cc:3037] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:10.436599 17335 raft_consensus.cc:2454] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 198d76df7d5040a9ae084699952e4356 in term 1.
I20260923 18:00:10.436599 17475 raft_consensus.cc:2454] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 198d76df7d5040a9ae084699952e4356 in term 1.
I20260923 18:00:10.437008 17127 leader_election.cc:302] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 198d76df7d5040a9ae084699952e4356, d1bd1a157a114ae1b2e8b84cfe4ef0e0; no voters:
I20260923 18:00:10.437212 17552 raft_consensus.cc:2781] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:10.437469 17552 raft_consensus.cc:686] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 1 LEADER]: Becoming Leader. State: Replica: 198d76df7d5040a9ae084699952e4356, State: Running, Role: LEADER
I20260923 18:00:10.437733 17552 consensus_queue.cc:237] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.440137 17038 catalog_manager.cc:5495] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 reported cstate change: term changed from 0 to 1, leader changed from <none> to 198d76df7d5040a9ae084699952e4356 (127.15.139.129). New cstate: current_term: 1 leader_uuid: "198d76df7d5040a9ae084699952e4356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } health_report { overall_health: UNKNOWN } } }
Sep 23 18:00:10 dist-test-slave-vr57 krb5kdc[16993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186409, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:10.487031 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.477468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38884 (local address 127.15.139.190:38651)
0923 18:00:10.477592 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.477596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.477781 (+ 185us) server_negotiation.cc:407] Connection header received
0923 18:00:10.477927 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.477929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.477969 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.478040 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:10.478951 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.479483 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.480722 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.480954 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.481147 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.483676 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:10.483679 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:10.483681 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:10.483700 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:10.485209 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:10.485701 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:10.485707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:10.485709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:10.485790 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:10.486080 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:10.486084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:10.486087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:10.486262 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:10.486369 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.486701 (+ 332us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.486842 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:00:10.487047 17582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.477290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:38651 (local address 127.0.0.1:38884)
0923 18:00:10.477672 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.477690 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.477826 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.478107 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.478112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.478676 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:10.478800 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.478808 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.479671 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.479675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.480578 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.480589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.481156 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.481159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.481246 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.481863 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:10.481874 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:10.483466 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:10.485325 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:10.485334 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:10.485340 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:10.485600 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:10.485881 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:10.485886 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:10.485888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:10.485967 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:10.486369 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:10.486377 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:10.486497 (+ 120us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.486654 (+ 157us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.486911 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":89,"thread_start_us":143,"threads_started":1}
I20260923 18:00:10.491091 17038 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:38884:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 18:00:10.491854 17038 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:10.497051 17315 tablet_service.cc:1467] Processing CreateTablet for tablet d052790e49a541dfb384783fbb5455a6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=01cd94123a9f49c3be17762df2d08fd7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 18:00:10.497051 17175 tablet_service.cc:1467] Processing CreateTablet for tablet d052790e49a541dfb384783fbb5455a6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=01cd94123a9f49c3be17762df2d08fd7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 18:00:10.497457 17175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d052790e49a541dfb384783fbb5455a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:10.497457 17315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d052790e49a541dfb384783fbb5455a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:10.499910 17455 tablet_service.cc:1467] Processing CreateTablet for tablet d052790e49a541dfb384783fbb5455a6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=01cd94123a9f49c3be17762df2d08fd7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 18:00:10.500269 17455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d052790e49a541dfb384783fbb5455a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:10.500453 17546 tablet_bootstrap.cc:492] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: Bootstrap starting.
I20260923 18:00:10.500453 17545 tablet_bootstrap.cc:492] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356: Bootstrap starting.
I20260923 18:00:10.501415 17545 tablet_bootstrap.cc:654] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:10.501415 17546 tablet_bootstrap.cc:654] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:10.503065 17545 tablet_bootstrap.cc:492] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356: No bootstrap required, opened a new log
I20260923 18:00:10.503142 17545 ts_tablet_manager.cc:1397] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:10.503283 17546 tablet_bootstrap.cc:492] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: No bootstrap required, opened a new log
I20260923 18:00:10.503355 17546 ts_tablet_manager.cc:1397] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260923 18:00:10.503397 17544 tablet_bootstrap.cc:492] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Bootstrap starting.
I20260923 18:00:10.503674 17545 raft_consensus.cc:348] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.503782 17545 raft_consensus.cc:374] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:10.503821 17545 raft_consensus.cc:729] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 198d76df7d5040a9ae084699952e4356, State: Initialized, Role: FOLLOWER
I20260923 18:00:10.503808 17546 raft_consensus.cc:348] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.503921 17546 raft_consensus.cc:374] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:10.503952 17546 raft_consensus.cc:729] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15d5429fbf68488c84799bc2496d5b38, State: Initialized, Role: FOLLOWER
I20260923 18:00:10.503952 17545 consensus_queue.cc:260] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.504076 17546 consensus_queue.cc:260] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.504215 17545 ts_tablet_manager.cc:1428] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:10.504328 17546 ts_tablet_manager.cc:1428] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:10.504391 17544 tablet_bootstrap.cc:654] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:10.504392 17552 tablet_replica.cc:439] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } }
I20260923 18:00:10.504525 17552 tablet_replica.cc:442] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:10.504832 17551 tablet_replica.cc:439] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } }
I20260923 18:00:10.504933 17551 tablet_replica.cc:442] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:10.506999 17544 tablet_bootstrap.cc:492] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0: No bootstrap required, opened a new log
I20260923 18:00:10.507084 17544 ts_tablet_manager.cc:1397] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260923 18:00:10.507453 17544 raft_consensus.cc:348] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.507580 17544 raft_consensus.cc:374] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:10.507611 17544 raft_consensus.cc:729] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bd1a157a114ae1b2e8b84cfe4ef0e0, State: Initialized, Role: FOLLOWER
I20260923 18:00:10.507747 17544 consensus_queue.cc:260] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.507939 17554 tablet_replica.cc:439] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } }
I20260923 18:00:10.508044 17554 tablet_replica.cc:442] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:10.509371 17544 ts_tablet_manager.cc:1428] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260923 18:00:10.588811 17522 tablet.cc:2369] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:00:10.610042 17382 tablet.cc:2369] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:00:10.695237 17551 raft_consensus.cc:482] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:10.695394 17551 raft_consensus.cc:504] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.696625 17551 leader_election.cc:288] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 198d76df7d5040a9ae084699952e4356 (127.15.139.129:38383), d1bd1a157a114ae1b2e8b84cfe4ef0e0 (127.15.139.131:46185)
I20260923 18:00:10.700670 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.696958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:42939 (local address 127.15.139.129:38383)
0923 18:00:10.697113 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.697119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.697301 (+ 182us) server_negotiation.cc:407] Connection header received
0923 18:00:10.697472 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.697475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.697519 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.697605 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:10.697886 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.698396 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.699542 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.700067 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.700197 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.700248 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.700443 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.700538 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 18:00:10.700661 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.697204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:59443 (local address 127.15.139.131:46185)
0923 18:00:10.697357 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.697362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.697377 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:00:10.697524 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.697531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.697571 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.697655 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:10.697907 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.698394 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.699605 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.700029 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.700160 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.700214 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.700463 (+ 249us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.700537 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 18:00:10.700693 17586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.696803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:38383 (local address 127.15.139.130:42939)
0923 18:00:10.697210 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.697226 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.697336 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.697615 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.697618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.697633 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:10.697756 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.697769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.698507 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.698512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.699394 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.699402 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.700197 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.700201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.700295 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.700315 (+ 20us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.700422 (+ 107us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.700517 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":39,"thread_start_us":74,"threads_started":1}
I20260923 18:00:10.700695 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.696870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:46185 (local address 127.15.139.130:59443)
0923 18:00:10.697281 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.697296 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.697393 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.697667 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.697670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.697686 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:10.697805 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.697810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.698505 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.698511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.699431 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.699441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.700161 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.700164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.700261 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.700291 (+ 30us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.700442 (+ 151us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.700549 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":91,"thread_start_us":82,"threads_started":1}
I20260923 18:00:10.701251 17195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d052790e49a541dfb384783fbb5455a6" candidate_uuid: "15d5429fbf68488c84799bc2496d5b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "198d76df7d5040a9ae084699952e4356" is_pre_election: true
I20260923 18:00:10.701319 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d052790e49a541dfb384783fbb5455a6" candidate_uuid: "15d5429fbf68488c84799bc2496d5b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" is_pre_election: true
I20260923 18:00:10.701434 17195 raft_consensus.cc:2454] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15d5429fbf68488c84799bc2496d5b38 in term 0.
I20260923 18:00:10.701462 17475 raft_consensus.cc:2454] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15d5429fbf68488c84799bc2496d5b38 in term 0.
I20260923 18:00:10.701778 17266 leader_election.cc:302] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15d5429fbf68488c84799bc2496d5b38, 198d76df7d5040a9ae084699952e4356; no voters:
I20260923 18:00:10.702057 17551 raft_consensus.cc:2781] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:10.702119 17551 raft_consensus.cc:482] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:10.702148 17551 raft_consensus.cc:3037] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:10.703190 17551 raft_consensus.cc:504] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.703606 17551 leader_election.cc:288] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [CANDIDATE]: Term 1 election: Requested vote from peers 198d76df7d5040a9ae084699952e4356 (127.15.139.129:38383), d1bd1a157a114ae1b2e8b84cfe4ef0e0 (127.15.139.131:46185)
I20260923 18:00:10.703896 17195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d052790e49a541dfb384783fbb5455a6" candidate_uuid: "15d5429fbf68488c84799bc2496d5b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "198d76df7d5040a9ae084699952e4356"
I20260923 18:00:10.703967 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d052790e49a541dfb384783fbb5455a6" candidate_uuid: "15d5429fbf68488c84799bc2496d5b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0"
I20260923 18:00:10.704015 17195 raft_consensus.cc:3037] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:10.704109 17475 raft_consensus.cc:3037] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:10.704953 17195 raft_consensus.cc:2454] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15d5429fbf68488c84799bc2496d5b38 in term 1.
I20260923 18:00:10.704993 17475 raft_consensus.cc:2454] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15d5429fbf68488c84799bc2496d5b38 in term 1.
I20260923 18:00:10.705195 17266 leader_election.cc:302] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15d5429fbf68488c84799bc2496d5b38, 198d76df7d5040a9ae084699952e4356; no voters:
I20260923 18:00:10.705356 17551 raft_consensus.cc:2781] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:10.705575 17551 raft_consensus.cc:686] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [term 1 LEADER]: Becoming Leader. State: Replica: 15d5429fbf68488c84799bc2496d5b38, State: Running, Role: LEADER
I20260923 18:00:10.705826 17551 consensus_queue.cc:237] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } }
I20260923 18:00:10.707243 17588 tablet_replica.cc:439] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: TxnStatusTablet state changed. Reason: New leader 15d5429fbf68488c84799bc2496d5b38. Latest consensus state: current_term: 1 leader_uuid: "15d5429fbf68488c84799bc2496d5b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } }
I20260923 18:00:10.707348 17588 tablet_replica.cc:442] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: This TxnStatusTablet replica's current role is: LEADER
I20260923 18:00:10.707619 17590 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 18:00:10.707773 17590 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 18:00:10.708487 17035 catalog_manager.cc:5495] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15d5429fbf68488c84799bc2496d5b38 (127.15.139.130). New cstate: current_term: 1 leader_uuid: "15d5429fbf68488c84799bc2496d5b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } health_report { overall_health: UNKNOWN } } }
I20260923 18:00:10.752674 17543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.748170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37368 (local address 127.15.139.130:44535)
0923 18:00:10.748308 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.748314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.748330 (+ 16us) server_negotiation.cc:407] Connection header received
0923 18:00:10.748540 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.748544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.748611 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.748714 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:10.748958 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.749740 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.750860 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.751207 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.751312 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.751758 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:10.752238 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:10.752380 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.752457 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.752517 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 18:00:10.752674 17582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.748010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:44535 (local address 127.0.0.1:37368)
0923 18:00:10.748256 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.748282 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.748383 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.748744 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.748746 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.748753 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 18:00:10.748852 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.748856 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.749856 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.749859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.750741 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.750751 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.751360 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.751364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.751483 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.751577 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 18:00:10.752351 (+ 774us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 18:00:10.752368 (+ 17us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.752447 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.752540 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260923 18:00:10.756471 17195 raft_consensus.cc:1261] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356 [term 1 FOLLOWER]: Refusing update from remote peer 15d5429fbf68488c84799bc2496d5b38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:10.756657 17475 raft_consensus.cc:1261] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 1 FOLLOWER]: Refusing update from remote peer 15d5429fbf68488c84799bc2496d5b38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:10.757105 17588 consensus_queue.cc:1035] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:10.757284 17551 consensus_queue.cc:1035] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:10.758901 17552 tablet_replica.cc:439] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356: TxnStatusTablet state changed. Reason: New leader 15d5429fbf68488c84799bc2496d5b38. Latest consensus state: current_term: 1 leader_uuid: "15d5429fbf68488c84799bc2496d5b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } }
I20260923 18:00:10.759022 17552 tablet_replica.cc:442] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:10.760123 17554 tablet_replica.cc:439] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0: TxnStatusTablet state changed. Reason: New leader 15d5429fbf68488c84799bc2496d5b38. Latest consensus state: current_term: 1 leader_uuid: "15d5429fbf68488c84799bc2496d5b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } }
I20260923 18:00:10.760267 17554 tablet_replica.cc:442] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:10.762161 17551 tablet_replica.cc:439] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "15d5429fbf68488c84799bc2496d5b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } }
I20260923 18:00:10.762326 17551 tablet_replica.cc:442] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: This TxnStatusTablet replica's current role is: LEADER
I20260923 18:00:10.762238 17588 tablet_replica.cc:439] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "15d5429fbf68488c84799bc2496d5b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } }
I20260923 18:00:10.762800 17588 tablet_replica.cc:442] T d052790e49a541dfb384783fbb5455a6 P 15d5429fbf68488c84799bc2496d5b38: This TxnStatusTablet replica's current role is: LEADER
I20260923 18:00:10.762821 17552 tablet_replica.cc:439] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "15d5429fbf68488c84799bc2496d5b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } }
I20260923 18:00:10.762961 17552 tablet_replica.cc:442] T d052790e49a541dfb384783fbb5455a6 P 198d76df7d5040a9ae084699952e4356: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:10.764458 17554 tablet_replica.cc:439] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "15d5429fbf68488c84799bc2496d5b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "198d76df7d5040a9ae084699952e4356" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38383 } } peers { permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 } } peers { permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 } } }
I20260923 18:00:10.764585 17554 tablet_replica.cc:442] T d052790e49a541dfb384783fbb5455a6 P d1bd1a157a114ae1b2e8b84cfe4ef0e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:10.771881 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.767554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58920 (local address 127.15.139.129:38383)
0923 18:00:10.767660 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.767664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.767811 (+ 147us) server_negotiation.cc:407] Connection header received
0923 18:00:10.767939 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.767942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.768017 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.768126 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:10.768346 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.769120 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.770024 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.770369 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.770475 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.770761 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:10.771396 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:10.771544 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.771676 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.771739 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 18:00:10.783766 17543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.779710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:60067 (local address 127.15.139.130:44535)
0923 18:00:10.779857 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.779862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.780035 (+ 173us) server_negotiation.cc:407] Connection header received
0923 18:00:10.780173 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.780175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.780221 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.780287 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:10.780628 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.781265 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.782524 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.782775 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.782951 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.783276 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:10.783356 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:10.783432 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.783612 (+ 180us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.783663 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260923 18:00:10.783823 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.779644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:44535 (local address 127.15.139.129:60067)
0923 18:00:10.779944 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.779959 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.780071 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.780316 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.780319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.780327 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 18:00:10.780460 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.780466 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.781390 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.781396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.782391 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.782402 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.782973 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.782977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.783084 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.783136 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 18:00:10.783446 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 18:00:10.783453 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.783552 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.783634 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":116,"threads_started":1}
I20260923 18:00:10.791471 17474 raft_consensus.cc:1261] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 1 FOLLOWER]: Refusing update from remote peer 198d76df7d5040a9ae084699952e4356: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:10.791586 17335 raft_consensus.cc:1261] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38 [term 1 FOLLOWER]: Refusing update from remote peer 198d76df7d5040a9ae084699952e4356: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:10.792084 17575 consensus_queue.cc:1035] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15d5429fbf68488c84799bc2496d5b38" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:10.792337 17552 consensus_queue.cc:1035] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1bd1a157a114ae1b2e8b84cfe4ef0e0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:10.797541 17558 mvcc.cc:205] Tried to move back new op lower bound from 7332603538597142528 to 7332603537155252224. Current Snapshot: MvccSnapshot[applied={T|T < 7332603538597142528}]
I20260923 18:00:10.804236 17557 mvcc.cc:205] Tried to move back new op lower bound from 7332603538597142528 to 7332603537155252224. Current Snapshot: MvccSnapshot[applied={T|T < 7332603538597142528}]
I20260923 18:00:10.820250 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.816360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:47425 (local address 127.15.139.129:38383)
0923 18:00:10.816463 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.816467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.816481 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:00:10.816555 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.816559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.816608 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.816691 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:10.816908 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.817691 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.818940 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.819277 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.819384 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.819753 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:10.819838 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:10.819915 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.820049 (+ 134us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.820103 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 18:00:10.820396 17603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.816044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:38383 (local address 127.15.139.130:47425)
0923 18:00:10.816366 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:10.816383 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:10.816473 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:10.816696 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:10.816699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:10.816705 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 18:00:10.816805 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.816810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.817796 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.817800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.818708 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:10.818722 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.819413 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.819417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:10.819491 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.819598 (+ 107us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 18:00:10.819941 (+ 343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 18:00:10.819947 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 18:00:10.820015 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 18:00:10.820181 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":95,"threads_started":1}
I20260923 18:00:10.850689 17035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:10.850881 17035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:10.853454 17035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 057c2faa66a74c6d9f6b878aba4a73a9: Sending DeleteTablet for 3 replicas of tablet ef9982b88f534394bbadcf23b0487b1f
I20260923 18:00:10.854002 17175 tablet_service.cc:1514] Processing DeleteTablet for tablet ef9982b88f534394bbadcf23b0487b1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:10 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:58910
I20260923 18:00:10.854034 17315 tablet_service.cc:1514] Processing DeleteTablet for tablet ef9982b88f534394bbadcf23b0487b1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:10 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:37366
I20260923 18:00:10.854197 17568 tablet_replica.cc:330] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356: stopping tablet replica
I20260923 18:00:10.854218 17569 tablet_replica.cc:330] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38: stopping tablet replica
I20260923 18:00:10.854305 17568 raft_consensus.cc:2229] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:10.854342 17569 raft_consensus.cc:2229] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:10.854480 17569 raft_consensus.cc:2258] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:10.854610 17568 raft_consensus.cc:2258] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:10.855396 17569 ts_tablet_manager.cc:1905] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:10.855437 17455 tablet_service.cc:1514] Processing DeleteTablet for tablet ef9982b88f534394bbadcf23b0487b1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:10 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:35762
I20260923 18:00:10.855643 17567 tablet_replica.cc:330] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0: stopping tablet replica
I20260923 18:00:10.855753 17567 raft_consensus.cc:2229] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:10.855868 17567 raft_consensus.cc:2258] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:10.856873 17568 ts_tablet_manager.cc:1905] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:10.858441 17569 ts_tablet_manager.cc:1918] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:10.858506 17569 log.cc:1196] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/ef9982b88f534394bbadcf23b0487b1f
I20260923 18:00:10.858749 17569 ts_tablet_manager.cc:1939] T ef9982b88f534394bbadcf23b0487b1f P 15d5429fbf68488c84799bc2496d5b38: Deleting consensus metadata
I20260923 18:00:10.859565 17023 catalog_manager.cc:4841] TS 15d5429fbf68488c84799bc2496d5b38 (127.15.139.130:44535): tablet ef9982b88f534394bbadcf23b0487b1f (table test-table [id=92e1a133920343449b92953879ff5d5b]) successfully deleted
I20260923 18:00:10.860224 17568 ts_tablet_manager.cc:1918] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:10.861423 17568 log.cc:1196] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/ef9982b88f534394bbadcf23b0487b1f
I20260923 18:00:10.861788 17568 ts_tablet_manager.cc:1939] T ef9982b88f534394bbadcf23b0487b1f P 198d76df7d5040a9ae084699952e4356: Deleting consensus metadata
I20260923 18:00:10.862591 17022 catalog_manager.cc:4841] TS 198d76df7d5040a9ae084699952e4356 (127.15.139.129:38383): tablet ef9982b88f534394bbadcf23b0487b1f (table test-table [id=92e1a133920343449b92953879ff5d5b]) successfully deleted
I20260923 18:00:10.863441 17567 ts_tablet_manager.cc:1905] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:10.866217 17567 ts_tablet_manager.cc:1918] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:10.866285 17567 log.cc:1196] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/ef9982b88f534394bbadcf23b0487b1f
I20260923 18:00:10.866555 17567 ts_tablet_manager.cc:1939] T ef9982b88f534394bbadcf23b0487b1f P d1bd1a157a114ae1b2e8b84cfe4ef0e0: Deleting consensus metadata
I20260923 18:00:10.867256 17023 catalog_manager.cc:4841] TS d1bd1a157a114ae1b2e8b84cfe4ef0e0 (127.15.139.131:46185): tablet ef9982b88f534394bbadcf23b0487b1f (table test-table [id=92e1a133920343449b92953879ff5d5b]) successfully deleted
Sep 23 18:00:10 dist-test-slave-vr57 krb5kdc[16993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186410, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.139.129@KRBTEST.COM
I20260923 18:00:10.887328 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.868344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58930 (local address 127.15.139.129:38383)
0923 18:00:10.868529 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.868534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.869193 (+ 659us) server_negotiation.cc:407] Connection header received
0923 18:00:10.869399 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.869402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.869452 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.869506 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:10.870470 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.871248 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.872511 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.872848 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.872923 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.882280 (+ 9357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:10.882296 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:10.882313 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:10.882355 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:10.885126 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:10.885722 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:10.885729 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:10.885731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:10.885838 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:10.886156 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:10.886159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:10.886162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:10.886530 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:10.886824 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.886982 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.887143 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260923 18:00:10.887864 17235 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58930
I20260923 18:00:10.899112 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:10.889930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38894 (local address 127.15.139.190:38651)
0923 18:00:10.890049 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:10.890054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:10.890164 (+ 110us) server_negotiation.cc:407] Connection header received
0923 18:00:10.890264 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:10.890268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:10.890314 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:10.890396 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:10.891188 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.892041 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.893174 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:10.893606 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:10.893729 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:10.895537 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:10.895542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:10.895545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:10.895576 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:10.897288 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:10.897779 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:10.897783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:10.897785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:10.897870 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:10.898152 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:10.898155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:10.898156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:10.898357 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:10.898467 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:10.898824 (+ 357us) server_negotiation.cc:299] Negotiation successful
0923 18:00:10.898949 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260923 18:00:10.900097 17035 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38894
I20260923 18:00:10.901543 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 17110
I20260923 18:00:10.908681 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 17250
I20260923 18:00:10.915921 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 17390
I20260923 18:00:10.922644 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 17006
2026-09-23T18:00:10Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4161 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:10 dist-test-slave-vr57 krb5kdc[17613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:10 dist-test-slave-vr57 krb5kdc[17613](info): set up 2 sockets
Sep 23 18:00:10 dist-test-slave-vr57 krb5kdc[17613](info): commencing operation
krb5kdc: starting...
W20260923 18:00:11.971987 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:12 dist-test-slave-vr57 krb5kdc[17613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:00:12.104575 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40067
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:40067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:12.226567 17626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:12.226964 17626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:12.227061 17626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:12.231434 17626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:12.231597 17626 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 18:00:12.231668 17626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:12.231737 17626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:12.231782 17626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:12.231823 17626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:12.237660 17626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:40067
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:12.238930 17626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:12.240206 17626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:12.249547 17632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:12.249598 17631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:12.249522 17634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:12.249941 17626 server_base.cc:1034] running on GCE node
I20260923 18:00:12.250598 17626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:12.251892 17626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:12.253110 17626 hybrid_clock.cc:648] HybridClock initialized: now 1790186412253078 us; error 45 us; skew 500 ppm
Sep 23 18:00:12 dist-test-slave-vr57 krb5kdc[17613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186412, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:12.256124 17626 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:00:12.257135 17626 webserver.cc:460] Webserver started at http://127.15.139.190:42829/ using document root <none> and password file <none>
I20260923 18:00:12.257732 17626 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:12.257829 17626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:12.258088 17626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:12.260387 17626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "535877aafb4a49688ef43c18236f5e99"
format_stamp: "Formatted at 2026-09-23 18:00:12 on dist-test-slave-vr57"
server_key: "e6be10fdf5a101d594010a00ad162f48"
server_key_iv: "eeb2ebe7a74d3a2c15b5eeddb41797ef"
server_key_version: "encryptionkey@0"
I20260923 18:00:12.260987 17626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "535877aafb4a49688ef43c18236f5e99"
format_stamp: "Formatted at 2026-09-23 18:00:12 on dist-test-slave-vr57"
server_key: "e6be10fdf5a101d594010a00ad162f48"
server_key_iv: "eeb2ebe7a74d3a2c15b5eeddb41797ef"
server_key_version: "encryptionkey@0"
I20260923 18:00:12.265113 17626 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:00:12.267750 17641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:12.268985 17626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260923 18:00:12.269163 17626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "535877aafb4a49688ef43c18236f5e99"
format_stamp: "Formatted at 2026-09-23 18:00:12 on dist-test-slave-vr57"
server_key: "e6be10fdf5a101d594010a00ad162f48"
server_key_iv: "eeb2ebe7a74d3a2c15b5eeddb41797ef"
server_key_version: "encryptionkey@0"
I20260923 18:00:12.269285 17626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:12.281826 17626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:12.285243 17626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:12.285459 17626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:12.293737 17626 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:40067
I20260923 18:00:12.293762 17703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:40067 every 8 connection(s)
I20260923 18:00:12.294725 17626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:12.297749 17704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:12.301573 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 17626
I20260923 18:00:12.301695 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:12.301985 15918 external_mini_cluster.cc:1428] Setting key cc943ad7df8b2bffbe2b202a873c0562
I20260923 18:00:12.304226 17704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99: Bootstrap starting.
I20260923 18:00:12.306644 17704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:12.307476 17704 log.cc:824] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:12.309968 17704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99: No bootstrap required, opened a new log
Sep 23 18:00:12 dist-test-slave-vr57 krb5kdc[17613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:12.314461 17704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535877aafb4a49688ef43c18236f5e99" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40067 } }
I20260923 18:00:12.314755 17704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:12.314845 17704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 535877aafb4a49688ef43c18236f5e99, State: Initialized, Role: FOLLOWER
I20260923 18:00:12.315379 17704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535877aafb4a49688ef43c18236f5e99" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40067 } }
I20260923 18:00:12.315557 17704 raft_consensus.cc:388] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:12.315629 17704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:12.315729 17704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:12.316711 17704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535877aafb4a49688ef43c18236f5e99" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40067 } }
I20260923 18:00:12.317081 17704 leader_election.cc:302] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 535877aafb4a49688ef43c18236f5e99; no voters:
I20260923 18:00:12.317518 17704 leader_election.cc:288] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:12.317680 17709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:12.317930 17709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [term 1 LEADER]: Becoming Leader. State: Replica: 535877aafb4a49688ef43c18236f5e99, State: Running, Role: LEADER
I20260923 18:00:12.318255 17709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535877aafb4a49688ef43c18236f5e99" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40067 } }
I20260923 18:00:12.318876 17704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:12.318982 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:12.303680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49742 (local address 127.15.139.190:40067)
0923 18:00:12.304175 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:12.304191 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:12.304229 (+ 38us) server_negotiation.cc:407] Connection header received
0923 18:00:12.304960 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:12.304979 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:12.305352 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:12.305720 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:12.306558 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.307969 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.309312 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.309826 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.310112 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:12.313084 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:12.313089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:12.313105 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:12.313135 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:12.315276 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:12.315929 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:12.315936 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:12.315943 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:12.316077 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:12.316394 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:12.316397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:12.316399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:12.316899 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:12.317123 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:12.317465 (+ 342us) server_negotiation.cc:299] Negotiation successful
0923 18:00:12.317908 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":162,"threads_started":1}
I20260923 18:00:12.319444 17709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 535877aafb4a49688ef43c18236f5e99. Latest consensus state: current_term: 1 leader_uuid: "535877aafb4a49688ef43c18236f5e99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535877aafb4a49688ef43c18236f5e99" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40067 } } }
I20260923 18:00:12.319577 17709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:12.319957 17714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:12.320427 17711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "535877aafb4a49688ef43c18236f5e99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535877aafb4a49688ef43c18236f5e99" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40067 } } }
I20260923 18:00:12.320581 17711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:12.323191 17714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:12.328521 17714 catalog_manager.cc:1347] Generated new cluster ID: cb2024dab62642a7a88a6b719b597b32
I20260923 18:00:12.328598 17714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:12.345155 17714 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:12.346184 17714 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:12.350023 17714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99: Generated new TSK 0
I20260923 18:00:12.350706 17714 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:00:12.384050 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40067
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:12.513592 17730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:12.513875 17730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:12.513922 17730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:12.518069 17730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:12.518143 17730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:12.518185 17730 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 18:00:12.518199 17730 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 18:00:12.518265 17730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:12.524219 17730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.139.190:40067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:12.525447 17730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:12.526582 17730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:12.537259 17735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:12.537436 17738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:12.537362 17736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:12.537575 17730 server_base.cc:1034] running on GCE node
I20260923 18:00:12.538007 17730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:12.538719 17730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:12.539954 17730 hybrid_clock.cc:648] HybridClock initialized: now 1790186412539933 us; error 31 us; skew 500 ppm
Sep 23 18:00:12 dist-test-slave-vr57 krb5kdc[17613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186412, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:12.543128 17730 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:00:12.544157 17730 webserver.cc:460] Webserver started at http://127.15.139.129:40555/ using document root <none> and password file <none>
I20260923 18:00:12.544770 17730 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:12.544832 17730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:12.545053 17730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:12.547202 17730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "b21c3ea4f71a4e11ad05d23ad00efc00"
format_stamp: "Formatted at 2026-09-23 18:00:12 on dist-test-slave-vr57"
server_key: "7ac3c45104e591b465be15071af52149"
server_key_iv: "a3bd9c0fe21118b465e15684f6729a52"
server_key_version: "encryptionkey@0"
I20260923 18:00:12.547832 17730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "b21c3ea4f71a4e11ad05d23ad00efc00"
format_stamp: "Formatted at 2026-09-23 18:00:12 on dist-test-slave-vr57"
server_key: "7ac3c45104e591b465be15071af52149"
server_key_iv: "a3bd9c0fe21118b465e15684f6729a52"
server_key_version: "encryptionkey@0"
I20260923 18:00:12.552016 17730 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260923 18:00:12.555267 17745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:12.556697 17730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:12.556843 17730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "b21c3ea4f71a4e11ad05d23ad00efc00"
format_stamp: "Formatted at 2026-09-23 18:00:12 on dist-test-slave-vr57"
server_key: "7ac3c45104e591b465be15071af52149"
server_key_iv: "a3bd9c0fe21118b465e15684f6729a52"
server_key_version: "encryptionkey@0"
I20260923 18:00:12.557001 17730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:12.573601 17730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:12.577082 17730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:12.577301 17730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:12.578878 17730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:12.578959 17730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:12.579038 17730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:12.579080 17730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 18:00:12 dist-test-slave-vr57 krb5kdc[17613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186412, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:12.590569 17730 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:35339
I20260923 18:00:12.591487 17730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:12.591683 17860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:35339 every 8 connection(s)
I20260923 18:00:12.593381 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 17730
I20260923 18:00:12.593482 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:12.593704 15918 external_mini_cluster.cc:1428] Setting key 50e9ee7b2ecfbb9e4f943f2d30df0b63
I20260923 18:00:12.593829 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:12.579765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:57593 (local address 127.15.139.190:40067)
0923 18:00:12.579917 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:12.579922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:12.581002 (+ 1080us) server_negotiation.cc:407] Connection header received
0923 18:00:12.582398 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:12.582402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:12.582462 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:12.582588 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:12.583900 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.584578 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.585816 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.586207 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.586321 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:12.589160 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:12.589163 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:12.589166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:12.589203 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:12.591180 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:12.591905 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:12.591909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:12.591911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:12.592047 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:12.592527 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:12.592530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:12.592532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:12.592752 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:12.592836 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:12.593486 (+ 650us) server_negotiation.cc:299] Negotiation successful
0923 18:00:12.593635 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 18:00:12.594970 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:12.580299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40067 (local address 127.15.139.129:57593)
0923 18:00:12.580804 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:12.580868 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:12.582132 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:12.582779 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:12.582794 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:12.583139 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:12.583748 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:12.583764 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.584694 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.584697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:12.585681 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:12.585692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.586368 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.586372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:12.586617 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:12.587282 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:12.587309 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:12.588980 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:12.591321 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:12.591327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:12.591351 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:12.591792 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:12.592175 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:12.592179 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:12.592181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:12.592382 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:12.592934 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:12.592940 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:12.593188 (+ 248us) client_negotiation.cc:769] Sending connection context
0923 18:00:12.593422 (+ 234us) client_negotiation.cc:240] Negotiation successful
0923 18:00:12.593791 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":100,"threads_started":1}
I20260923 18:00:12.596521 17861 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40067
I20260923 18:00:12.596681 17861 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:12.597410 17861 heartbeater.cc:511] Master 127.15.139.190:40067 requested a full tablet report, sending...
I20260923 18:00:12.599099 17658 ts_manager.cc:194] Registered new tserver with Master: b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129:35339)
I20260923 18:00:12.600893 17658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:57593
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
I20260923 18:00:12.608018 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:12.599317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:54257 (local address 127.15.139.190:40067)
0923 18:00:12.599439 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:12.599444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:12.599778 (+ 334us) server_negotiation.cc:407] Connection header received
0923 18:00:12.599922 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:12.599925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:12.599972 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:12.600072 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:12.600551 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.601128 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.602186 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.602626 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.602761 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:12.604077 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:12.604082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:12.604088 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:12.604115 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:12.605850 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:12.606269 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:12.606272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:12.606273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:12.606346 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:12.606826 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:12.606830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:12.606832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:12.607047 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:12.607160 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:12.607582 (+ 422us) server_negotiation.cc:299] Negotiation successful
0923 18:00:12.607767 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:00:12.608075 17868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:12.599225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40067 (local address 127.15.139.129:54257)
0923 18:00:12.599683 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:12.599696 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:12.599822 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:12.600088 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:12.600094 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:12.600328 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:12.600436 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:12.600442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.601260 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.601263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:12.602049 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:12.602059 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.602768 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.602771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:12.602863 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:12.603535 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:12.603551 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:12.603929 (+ 378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:12.605966 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:12.605968 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:12.605970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:12.606180 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:12.606505 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:12.606508 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:12.606510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:12.606627 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:12.607159 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:12.607161 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:12.607322 (+ 161us) client_negotiation.cc:769] Sending connection context
0923 18:00:12.607461 (+ 139us) client_negotiation.cc:240] Negotiation successful
0923 18:00:12.607668 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":100,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:00:12.626773 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40067
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:12.741119 17870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:12.741389 17870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:12.741437 17870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:12.745728 17870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:12.745800 17870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:12.745842 17870 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 18:00:12.745901 17870 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 18:00:12.745967 17870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:12.751444 17870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.139.190:40067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:12.752601 17870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:12.753648 17870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:12.763572 17878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:12.763572 17875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:12.763782 17876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:12.764108 17870 server_base.cc:1034] running on GCE node
I20260923 18:00:12.764539 17870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:12.765229 17870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:12.766441 17870 hybrid_clock.cc:648] HybridClock initialized: now 1790186412766426 us; error 40 us; skew 500 ppm
Sep 23 18:00:12 dist-test-slave-vr57 krb5kdc[17613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186412, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:12.769330 17870 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:00:12.770296 17870 webserver.cc:460] Webserver started at http://127.15.139.130:40233/ using document root <none> and password file <none>
I20260923 18:00:12.770869 17870 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:12.771004 17870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:12.771239 17870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:12.773610 17870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "92ea0212286942cebd7b611fc3a9b816"
format_stamp: "Formatted at 2026-09-23 18:00:12 on dist-test-slave-vr57"
server_key: "38bd11ffa55705b12e99658c36438123"
server_key_iv: "6d14afb2450d861f0ef05244168ff46c"
server_key_version: "encryptionkey@0"
I20260923 18:00:12.774232 17870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "92ea0212286942cebd7b611fc3a9b816"
format_stamp: "Formatted at 2026-09-23 18:00:12 on dist-test-slave-vr57"
server_key: "38bd11ffa55705b12e99658c36438123"
server_key_iv: "6d14afb2450d861f0ef05244168ff46c"
server_key_version: "encryptionkey@0"
I20260923 18:00:12.778065 17870 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:12.780700 17885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:12.782048 17870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:12.782217 17870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "92ea0212286942cebd7b611fc3a9b816"
format_stamp: "Formatted at 2026-09-23 18:00:12 on dist-test-slave-vr57"
server_key: "38bd11ffa55705b12e99658c36438123"
server_key_iv: "6d14afb2450d861f0ef05244168ff46c"
server_key_version: "encryptionkey@0"
I20260923 18:00:12.782339 17870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:12.793285 17870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:12.797014 17870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:12.797219 17870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:12.798795 17870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:12.798871 17870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:12.798934 17870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:12.798967 17870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 18:00:12 dist-test-slave-vr57 krb5kdc[17613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186412, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:12.810611 17870 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:42657
I20260923 18:00:12.810614 18000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:42657 every 8 connection(s)
I20260923 18:00:12.811789 17870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:12.813377 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:12.799744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:42787 (local address 127.15.139.190:40067)
0923 18:00:12.799875 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:12.799879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:12.800956 (+ 1077us) server_negotiation.cc:407] Connection header received
0923 18:00:12.802088 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:12.802092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:12.802157 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:12.802253 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:12.803861 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.804443 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.805620 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.805848 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.805981 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:12.809154 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:12.809158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:12.809161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:12.809194 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:12.810937 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:12.811454 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:12.811458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:12.811459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:12.811576 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:12.812143 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:12.812148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:12.812150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:12.812418 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:12.812530 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:12.813133 (+ 603us) server_negotiation.cc:299] Negotiation successful
0923 18:00:12.813237 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 18:00:12.813544 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 17870
I20260923 18:00:12.813661 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:12.813961 15918 external_mini_cluster.cc:1428] Setting key 12973bd58f7d2f9b04b34fa61c69ab09
I20260923 18:00:12.814365 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:12.800325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40067 (local address 127.15.139.130:42787)
0923 18:00:12.800806 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:12.800848 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:12.801782 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:12.802454 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:12.802461 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:12.802804 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:12.803679 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:12.803703 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.804582 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.804586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:12.805486 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:12.805494 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.806031 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.806036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:12.806240 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:12.807009 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:12.807033 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:12.808976 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:12.811066 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:12.811071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:12.811083 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:12.811327 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:12.811678 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:12.811680 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:12.811682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:12.811821 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:12.812586 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:12.812592 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:12.812884 (+ 292us) client_negotiation.cc:769] Sending connection context
0923 18:00:12.813045 (+ 161us) client_negotiation.cc:240] Negotiation successful
0923 18:00:12.813336 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":133,"threads_started":1}
I20260923 18:00:12.815886 18001 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40067
I20260923 18:00:12.816126 18001 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:12.816706 18001 heartbeater.cc:511] Master 127.15.139.190:40067 requested a full tablet report, sending...
I20260923 18:00:12.817824 17653 ts_manager.cc:194] Registered new tserver with Master: 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130:42657)
I20260923 18:00:12.818629 17653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:42787
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
I20260923 18:00:12.826700 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:12.818427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:57089 (local address 127.15.139.190:40067)
0923 18:00:12.818532 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:12.818536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:12.818550 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:00:12.818644 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:12.818650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:12.818698 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:12.818793 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:12.819343 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.820139 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.821476 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.821679 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.821761 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:12.823178 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:12.823181 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:12.823183 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:12.823207 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:12.824927 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:12.825409 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:12.825412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:12.825413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:12.825528 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:12.825859 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:12.825863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:12.825865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:12.826090 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:12.826183 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:12.826431 (+ 248us) server_negotiation.cc:299] Negotiation successful
0923 18:00:12.826570 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 18:00:12.826725 18008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:12.818113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40067 (local address 127.15.139.130:57089)
0923 18:00:12.818430 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:12.818448 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:12.818556 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:12.818812 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:12.818815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:12.819049 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:12.819189 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:12.819200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.820281 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.820287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:12.821337 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:12.821352 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:12.821805 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:12.821810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:12.821948 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:12.822534 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:12.822555 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:12.823027 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:12.825047 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:12.825052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:12.825057 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:12.825299 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:12.825646 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:12.825650 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:12.825654 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:12.825759 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:12.826180 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:12.826182 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:12.826255 (+ 73us) client_negotiation.cc:769] Sending connection context
0923 18:00:12.826335 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 18:00:12.826496 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:00:12.842782 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40067
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:12.960141 18010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:12.960438 18010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:12.960521 18010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:12.964840 18010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:12.964916 18010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:12.964962 18010 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 18:00:12.964980 18010 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 18:00:12.965072 18010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:12.970806 18010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.139.190:40067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:12.972031 18010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:12.973142 18010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:12.985594 18016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:12.985632 18018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:12.986397 18015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:12.986546 18010 server_base.cc:1034] running on GCE node
I20260923 18:00:12.987154 18010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:12.988059 18010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:12.989264 18010 hybrid_clock.cc:648] HybridClock initialized: now 1790186412989253 us; error 43 us; skew 500 ppm
Sep 23 18:00:12 dist-test-slave-vr57 krb5kdc[17613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186412, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:12.992429 18010 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:00:12.993429 18010 webserver.cc:460] Webserver started at http://127.15.139.131:44653/ using document root <none> and password file <none>
I20260923 18:00:12.994067 18010 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:12.994165 18010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:12.994412 18010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:12.996800 18010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "c308465e7d1743d7b030159e95bd1c30"
format_stamp: "Formatted at 2026-09-23 18:00:12 on dist-test-slave-vr57"
server_key: "d679c1b28a73196d36e12e0991a87a06"
server_key_iv: "b5a26a2144f3c833cd36099bef8f51e4"
server_key_version: "encryptionkey@0"
I20260923 18:00:12.997424 18010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "c308465e7d1743d7b030159e95bd1c30"
format_stamp: "Formatted at 2026-09-23 18:00:12 on dist-test-slave-vr57"
server_key: "d679c1b28a73196d36e12e0991a87a06"
server_key_iv: "b5a26a2144f3c833cd36099bef8f51e4"
server_key_version: "encryptionkey@0"
I20260923 18:00:13.001835 18010 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:13.004664 18025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:13.006424 18010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:13.006605 18010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "c308465e7d1743d7b030159e95bd1c30"
format_stamp: "Formatted at 2026-09-23 18:00:12 on dist-test-slave-vr57"
server_key: "d679c1b28a73196d36e12e0991a87a06"
server_key_iv: "b5a26a2144f3c833cd36099bef8f51e4"
server_key_version: "encryptionkey@0"
I20260923 18:00:13.006743 18010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:13.024927 18010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:13.028395 18010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:13.028613 18010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:13.030236 18010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:13.030311 18010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:13.030382 18010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:13.030437 18010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 18:00:13 dist-test-slave-vr57 krb5kdc[17613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186412, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:13.041594 18010 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:42297
I20260923 18:00:13.041771 18140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:42297 every 8 connection(s)
I20260923 18:00:13.042670 18010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:13.045661 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.031575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:59671 (local address 127.15.139.190:40067)
0923 18:00:13.031730 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.031734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.032838 (+ 1104us) server_negotiation.cc:407] Connection header received
0923 18:00:13.034259 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.034263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.034327 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.034427 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:13.036198 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.037108 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.038437 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.038764 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.038891 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.042032 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:13.042036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:13.042039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:13.042069 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:13.043867 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:13.044323 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:13.044326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:13.044327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:13.044434 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:13.044711 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:13.044713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:13.044715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:13.044899 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:13.045016 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.045424 (+ 408us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.045534 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 18:00:13.046536 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.032113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40067 (local address 127.15.139.131:59671)
0923 18:00:13.032630 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.032704 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.034027 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.034633 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.034642 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.035184 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:13.036024 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.036049 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.037252 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.037256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.038246 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.038255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.038949 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.038954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.039157 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.039916 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:13.039934 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:13.041843 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:13.043963 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:13.043967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:13.043976 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:13.044223 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:13.044500 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:13.044502 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:13.044504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:13.044626 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:13.044988 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:13.044993 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:13.045210 (+ 217us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.045356 (+ 146us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.045619 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":136,"threads_started":1}
I20260923 18:00:13.047776 18141 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40067
I20260923 18:00:13.047915 18141 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:13.048466 18141 heartbeater.cc:511] Master 127.15.139.190:40067 requested a full tablet report, sending...
I20260923 18:00:13.049535 17653 ts_manager.cc:194] Registered new tserver with Master: c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297)
I20260923 18:00:13.049710 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 18010
I20260923 18:00:13.049824 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:00:13.050199 15918 external_mini_cluster.cc:1428] Setting key fc53eb98a05933471ccb0423bb82502c
I20260923 18:00:13.050343 17653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:59671
I20260923 18:00:13.052269 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:00:13.057757 18147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.050164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40067 (local address 127.15.139.131:52601)
0923 18:00:13.050483 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.050500 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.050604 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.051086 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.051090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.051316 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:13.051486 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.051533 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.052251 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.052254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.052886 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.052895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.053360 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.053364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.053457 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.053987 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:13.054010 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:13.054305 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:13.056260 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:13.056265 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:13.056267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:13.056463 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:13.056754 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:13.056757 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:13.056759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:13.056841 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:13.057339 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:13.057341 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:13.057413 (+ 72us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.057495 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.057624 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260923 18:00:13.058007 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.050672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:52601 (local address 127.15.139.190:40067)
0923 18:00:13.050781 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.050787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.050800 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:00:13.050833 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.050836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.050879 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.050957 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:13.051618 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.052127 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.053013 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.053236 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.053345 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.054454 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:13.054459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:13.054463 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:13.054493 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:13.056150 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:13.056570 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:13.056575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:13.056577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:13.056657 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:13.056971 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:13.056976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:13.056978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:13.057237 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:13.057349 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.057586 (+ 237us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.057724 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Sep 23 18:00:13 dist-test-slave-vr57 krb5kdc[17613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 18:00:13 dist-test-slave-vr57 krb5kdc[17613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:13.081099 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.072628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49752 (local address 127.15.139.190:40067)
0923 18:00:13.072764 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.072768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.072781 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:00:13.072822 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.072825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.072863 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.073041 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:13.073799 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.074324 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.075185 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.075408 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.075546 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.078040 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:13.078043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:13.078045 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:13.078069 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:13.079393 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:13.079843 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:13.079846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:13.079848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:13.079937 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:13.080191 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:13.080193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:13.080194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:13.080356 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:13.080471 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.080851 (+ 380us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.080979 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 18:00:13.084563 17653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:00:13.088093 17653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:13.103184 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.096933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35339 (local address 127.0.0.1:55024)
0923 18:00:13.097585 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.097637 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.097766 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.098350 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.098356 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.098381 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:13.098544 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.098554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.100181 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.100185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.101766 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.101776 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.102633 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.102638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.102771 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.102810 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.102920 (+ 110us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.103047 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":135,"thread_start_us":76,"threads_started":1}
I20260923 18:00:13.103194 18159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.097088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55024 (local address 127.15.139.129:35339)
0923 18:00:13.097671 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.097682 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.097712 (+ 30us) server_negotiation.cc:407] Connection header received
0923 18:00:13.097935 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.097942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.098173 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.098722 (+ 549us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:13.098854 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.100040 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.101915 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.102459 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.102707 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.102844 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.102946 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.103036 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":98,"threads_started":1}
I20260923 18:00:13.103567 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.097766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47678 (local address 127.15.139.131:42297)
0923 18:00:13.098124 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.098139 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.098522 (+ 383us) server_negotiation.cc:407] Connection header received
0923 18:00:13.098605 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.098611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.098817 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.098976 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:13.099190 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.100040 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.101767 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.102580 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.102723 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.102858 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.103300 (+ 442us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.103398 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":135,"threads_started":1}
I20260923 18:00:13.103667 18161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.097218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.15.139.130:42657)
0923 18:00:13.098235 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.098246 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.098708 (+ 462us) server_negotiation.cc:407] Connection header received
0923 18:00:13.098788 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.098793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.098983 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.099129 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:13.099325 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.100473 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.101890 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.102667 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.102785 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.102883 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.103401 (+ 518us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.103512 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":918,"thread_start_us":104,"threads_started":1}
I20260923 18:00:13.103751 18160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.097183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:42297 (local address 127.0.0.1:47678)
0923 18:00:13.098271 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.098287 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.098406 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.098949 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.098952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.098968 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:13.099061 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.099070 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.100153 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.100158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.101641 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.101650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.102935 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.102939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.103066 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.103121 (+ 55us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.103179 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.103605 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":486,"thread_start_us":110,"threads_started":1}
I20260923 18:00:13.104020 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.096933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:42657 (local address 127.0.0.1:56344)
0923 18:00:13.098595 (+ 1662us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.098607 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.098692 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.099116 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.099121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.099139 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:13.099231 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.099239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.100616 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.100620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.101766 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.101776 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.102935 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.102939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.103070 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.103107 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.103179 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.103275 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1606,"mutex_wait_us":902,"thread_start_us":97,"threads_started":1}
I20260923 18:00:13.106777 17935 tablet_service.cc:1467] Processing CreateTablet for tablet da2c773ca4584e4ab58997f0d9577e43 (DEFAULT_TABLE table=test-table [id=14c0ae98a6ae483c9a4a879644326207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:13.106791 18075 tablet_service.cc:1467] Processing CreateTablet for tablet da2c773ca4584e4ab58997f0d9577e43 (DEFAULT_TABLE table=test-table [id=14c0ae98a6ae483c9a4a879644326207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:13.106789 17795 tablet_service.cc:1467] Processing CreateTablet for tablet da2c773ca4584e4ab58997f0d9577e43 (DEFAULT_TABLE table=test-table [id=14c0ae98a6ae483c9a4a879644326207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:13.107868 17795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2c773ca4584e4ab58997f0d9577e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:13.107870 18075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2c773ca4584e4ab58997f0d9577e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:13.107909 17935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2c773ca4584e4ab58997f0d9577e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:13.115239 18164 tablet_bootstrap.cc:492] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816: Bootstrap starting.
I20260923 18:00:13.116786 18165 tablet_bootstrap.cc:492] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30: Bootstrap starting.
I20260923 18:00:13.116843 18166 tablet_bootstrap.cc:492] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00: Bootstrap starting.
I20260923 18:00:13.117405 18164 tablet_bootstrap.cc:654] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:13.118153 18164 log.cc:824] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:13.119024 18165 tablet_bootstrap.cc:654] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:13.119096 18166 tablet_bootstrap.cc:654] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:13.119980 18165 log.cc:824] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:13.120360 18166 log.cc:824] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:13.120828 18164 tablet_bootstrap.cc:492] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816: No bootstrap required, opened a new log
I20260923 18:00:13.121031 18164 ts_tablet_manager.cc:1397] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260923 18:00:13.121874 18165 tablet_bootstrap.cc:492] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30: No bootstrap required, opened a new log
I20260923 18:00:13.122119 18165 ts_tablet_manager.cc:1397] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260923 18:00:13.124176 18164 raft_consensus.cc:348] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.124430 18164 raft_consensus.cc:374] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:13.124492 18164 raft_consensus.cc:729] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ea0212286942cebd7b611fc3a9b816, State: Initialized, Role: FOLLOWER
I20260923 18:00:13.124816 18166 tablet_bootstrap.cc:492] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00: No bootstrap required, opened a new log
I20260923 18:00:13.125034 18166 ts_tablet_manager.cc:1397] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260923 18:00:13.124991 18164 consensus_queue.cc:260] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.125679 18001 heartbeater.cc:503] Master 127.15.139.190:40067 was elected leader, sending a full tablet report...
I20260923 18:00:13.125578 18165 raft_consensus.cc:348] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.125816 18165 raft_consensus.cc:374] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:13.125865 18165 raft_consensus.cc:729] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c308465e7d1743d7b030159e95bd1c30, State: Initialized, Role: FOLLOWER
I20260923 18:00:13.126283 18165 consensus_queue.cc:260] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.127128 18164 ts_tablet_manager.cc:1428] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260923 18:00:13.127254 18165 ts_tablet_manager.cc:1428] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260923 18:00:13.127645 18141 heartbeater.cc:503] Master 127.15.139.190:40067 was elected leader, sending a full tablet report...
I20260923 18:00:13.128640 18166 raft_consensus.cc:348] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.128913 18166 raft_consensus.cc:374] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:13.128979 18166 raft_consensus.cc:729] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b21c3ea4f71a4e11ad05d23ad00efc00, State: Initialized, Role: FOLLOWER
I20260923 18:00:13.129594 18166 consensus_queue.cc:260] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.130210 17861 heartbeater.cc:503] Master 127.15.139.190:40067 was elected leader, sending a full tablet report...
I20260923 18:00:13.130641 18166 ts_tablet_manager.cc:1428] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260923 18:00:13.293762 18142 tablet.cc:2369] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:00:13.312955 18002 tablet.cc:2369] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:00:13.343009 17862 tablet.cc:2369] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:00:13.529953 18172 raft_consensus.cc:482] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:13.530218 18172 raft_consensus.cc:504] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.531447 18172 leader_election.cc:288] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130:42657), c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297)
I20260923 18:00:13.535712 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.531804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:42297 (local address 127.15.139.129:33157)
0923 18:00:13.532207 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.532219 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.532327 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.532609 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.532612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.532629 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:13.532771 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.532778 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.533660 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.533663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.534550 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.534558 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.535236 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.535243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.535333 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.535366 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.535435 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.535489 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":43,"thread_start_us":176,"threads_started":1}
I20260923 18:00:13.535725 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.532000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:33157 (local address 127.15.139.131:42297)
0923 18:00:13.532215 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.532220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.532302 (+ 82us) server_negotiation.cc:407] Connection header received
0923 18:00:13.532438 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.532444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.532502 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.532581 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:13.532953 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.533516 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.534668 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.535107 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.535204 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.535283 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.535474 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.535583 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260923 18:00:13.535772 18161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.531898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:40369 (local address 127.15.139.130:42657)
0923 18:00:13.532060 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.532067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.532526 (+ 459us) server_negotiation.cc:407] Connection header received
0923 18:00:13.532620 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.532623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.532671 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.532744 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:13.533034 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.533515 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.534767 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.535145 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.535284 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.535338 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.535581 (+ 243us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.535650 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 18:00:13.535866 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.531804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:42657 (local address 127.15.139.129:40369)
0923 18:00:13.532397 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.532412 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.532516 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.532766 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.532768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.532782 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:13.532913 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.532921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.533635 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.533640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.534641 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.534653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.535300 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.535305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.535403 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.535448 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.535560 (+ 112us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.535624 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":72,"thread_start_us":102,"threads_started":1}
I20260923 18:00:13.536454 17955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2c773ca4584e4ab58997f0d9577e43" candidate_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ea0212286942cebd7b611fc3a9b816" is_pre_election: true
I20260923 18:00:13.536463 18095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2c773ca4584e4ab58997f0d9577e43" candidate_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c308465e7d1743d7b030159e95bd1c30" is_pre_election: true
I20260923 18:00:13.536816 17955 raft_consensus.cc:2454] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b21c3ea4f71a4e11ad05d23ad00efc00 in term 0.
I20260923 18:00:13.536876 18095 raft_consensus.cc:2454] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b21c3ea4f71a4e11ad05d23ad00efc00 in term 0.
I20260923 18:00:13.537407 17749 leader_election.cc:302] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 92ea0212286942cebd7b611fc3a9b816, b21c3ea4f71a4e11ad05d23ad00efc00, c308465e7d1743d7b030159e95bd1c30; no voters:
I20260923 18:00:13.537703 18172 raft_consensus.cc:2781] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:13.537791 18172 raft_consensus.cc:482] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:13.537832 18172 raft_consensus.cc:3037] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:13.538815 18172 raft_consensus.cc:504] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.539207 18172 leader_election.cc:288] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [CANDIDATE]: Term 1 election: Requested vote from peers 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130:42657), c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297)
I20260923 18:00:13.539619 17955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2c773ca4584e4ab58997f0d9577e43" candidate_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ea0212286942cebd7b611fc3a9b816"
I20260923 18:00:13.539731 17955 raft_consensus.cc:3037] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:13.540733 17955 raft_consensus.cc:2454] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b21c3ea4f71a4e11ad05d23ad00efc00 in term 1.
I20260923 18:00:13.540998 18095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2c773ca4584e4ab58997f0d9577e43" candidate_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c308465e7d1743d7b030159e95bd1c30"
I20260923 18:00:13.541033 17748 leader_election.cc:302] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92ea0212286942cebd7b611fc3a9b816, b21c3ea4f71a4e11ad05d23ad00efc00; no voters:
I20260923 18:00:13.541126 18095 raft_consensus.cc:3037] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:13.541193 18172 raft_consensus.cc:2781] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:13.541419 18172 raft_consensus.cc:686] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 LEADER]: Becoming Leader. State: Replica: b21c3ea4f71a4e11ad05d23ad00efc00, State: Running, Role: LEADER
I20260923 18:00:13.541774 18172 consensus_queue.cc:237] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.542361 18095 raft_consensus.cc:2454] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b21c3ea4f71a4e11ad05d23ad00efc00 in term 1.
I20260923 18:00:13.545076 17653 catalog_manager.cc:5495] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 reported cstate change: term changed from 0 to 1, leader changed from <none> to b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129). New cstate: current_term: 1 leader_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } health_report { overall_health: UNKNOWN } } }
Sep 23 18:00:13 dist-test-slave-vr57 krb5kdc[17613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186412, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:13.686156 18182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.677220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40067 (local address 127.0.0.1:49760)
0923 18:00:13.677555 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.677570 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.677687 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.677964 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.677966 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.678385 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:13.678485 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.678491 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.679284 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.679287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.679995 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.680005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.680462 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.680465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.680531 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.681106 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:13.681118 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:13.682608 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:13.684515 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:13.684524 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:13.684530 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:13.684774 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:13.685134 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:13.685136 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:13.685137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:13.685215 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:13.685662 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:13.685667 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:13.685772 (+ 105us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.685890 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.686017 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":95,"threads_started":1}
I20260923 18:00:13.686388 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.677289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49760 (local address 127.15.139.190:40067)
0923 18:00:13.677618 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.677625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.677641 (+ 16us) server_negotiation.cc:407] Connection header received
0923 18:00:13.677782 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.677784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.677825 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.677905 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:13.678596 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.679164 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.680116 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.680351 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.680434 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.682756 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:13.682759 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:13.682760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:13.682781 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:13.684394 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:13.684945 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:13.684957 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:13.684958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:13.685043 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:13.685334 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:13.685337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:13.685338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:13.685532 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:13.685620 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.686033 (+ 413us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.686144 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":159,"threads_started":1}
I20260923 18:00:13.690245 17653 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:49760:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 18:00:13.691062 17653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:13.695986 17795 tablet_service.cc:1467] Processing CreateTablet for tablet 7353b23384434019b3662ea98fa58dd7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=403a6806af3043549af0ced65a494720]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 18:00:13.696024 18075 tablet_service.cc:1467] Processing CreateTablet for tablet 7353b23384434019b3662ea98fa58dd7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=403a6806af3043549af0ced65a494720]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 18:00:13.695987 17935 tablet_service.cc:1467] Processing CreateTablet for tablet 7353b23384434019b3662ea98fa58dd7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=403a6806af3043549af0ced65a494720]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 18:00:13.696768 18075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7353b23384434019b3662ea98fa58dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:13.696768 17795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7353b23384434019b3662ea98fa58dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:13.696815 17935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7353b23384434019b3662ea98fa58dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:13.700173 18184 tablet_bootstrap.cc:492] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: Bootstrap starting.
I20260923 18:00:13.700585 18185 tablet_bootstrap.cc:492] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30: Bootstrap starting.
I20260923 18:00:13.701300 18184 tablet_bootstrap.cc:654] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:13.701339 18186 tablet_bootstrap.cc:492] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00: Bootstrap starting.
I20260923 18:00:13.701602 18185 tablet_bootstrap.cc:654] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:13.702276 18186 tablet_bootstrap.cc:654] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:13.702643 18184 tablet_bootstrap.cc:492] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: No bootstrap required, opened a new log
I20260923 18:00:13.702736 18184 ts_tablet_manager.cc:1397] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:13.702775 18185 tablet_bootstrap.cc:492] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30: No bootstrap required, opened a new log
I20260923 18:00:13.702854 18185 ts_tablet_manager.cc:1397] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:13.703214 18184 raft_consensus.cc:348] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.703352 18184 raft_consensus.cc:374] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:13.703292 18185 raft_consensus.cc:348] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.703394 18185 raft_consensus.cc:374] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:13.703387 18184 raft_consensus.cc:729] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ea0212286942cebd7b611fc3a9b816, State: Initialized, Role: FOLLOWER
I20260923 18:00:13.703425 18186 tablet_bootstrap.cc:492] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00: No bootstrap required, opened a new log
I20260923 18:00:13.703434 18185 raft_consensus.cc:729] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c308465e7d1743d7b030159e95bd1c30, State: Initialized, Role: FOLLOWER
I20260923 18:00:13.703537 18186 ts_tablet_manager.cc:1397] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:13.703586 18184 consensus_queue.cc:260] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.703593 18185 consensus_queue.cc:260] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.703938 18184 ts_tablet_manager.cc:1428] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260923 18:00:13.703950 18185 ts_tablet_manager.cc:1428] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:13.704021 18186 raft_consensus.cc:348] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.704160 18186 raft_consensus.cc:374] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:13.704202 18186 raft_consensus.cc:729] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b21c3ea4f71a4e11ad05d23ad00efc00, State: Initialized, Role: FOLLOWER
I20260923 18:00:13.704339 18186 consensus_queue.cc:260] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.704638 18186 ts_tablet_manager.cc:1428] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:13.704931 18191 tablet_replica.cc:439] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } } }
I20260923 18:00:13.705075 18191 tablet_replica.cc:442] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:13.705222 18190 tablet_replica.cc:439] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } } }
I20260923 18:00:13.705361 18190 tablet_replica.cc:442] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:13.705431 18172 tablet_replica.cc:439] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } } }
I20260923 18:00:13.705580 18172 tablet_replica.cc:442] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:13.745797 18190 raft_consensus.cc:482] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:13.745975 18190 raft_consensus.cc:504] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.747357 18190 leader_election.cc:288] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129:35339), c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297)
I20260923 18:00:13.752127 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.747614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:42297 (local address 127.15.139.130:38795)
0923 18:00:13.747989 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.748027 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.748137 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.748472 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.748475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.748488 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:13.748972 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.748978 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.749796 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.749799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.750925 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.750932 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.751722 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.751725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.751812 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.751841 (+ 29us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.751965 (+ 124us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.752017 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":133,"threads_started":1}
I20260923 18:00:13.752140 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.747721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:38795 (local address 127.15.139.131:42297)
0923 18:00:13.747859 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.747864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.748095 (+ 231us) server_negotiation.cc:407] Connection header received
0923 18:00:13.748233 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.748236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.748276 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.748366 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:13.749087 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.749678 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.751034 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.751629 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.751732 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.751785 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.751930 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.752002 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 18:00:13.752156 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.747661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:46341 (local address 127.15.139.129:35339)
0923 18:00:13.748002 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.748019 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.748037 (+ 18us) server_negotiation.cc:407] Connection header received
0923 18:00:13.748185 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.748189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.748245 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.748358 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:13.748667 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.749330 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.750848 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.751526 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.751668 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.751733 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.751926 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.752013 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":92,"threads_started":1}
I20260923 18:00:13.752131 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.747568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35339 (local address 127.15.139.130:46341)
0923 18:00:13.747937 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.747953 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.748084 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.748390 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.748393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.748410 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:13.748544 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.748550 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.749462 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.749464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.750717 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.750726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.751644 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.751647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.751764 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.751797 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.751879 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.751956 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":16,"thread_start_us":122,"threads_started":1}
I20260923 18:00:13.752849 18095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7353b23384434019b3662ea98fa58dd7" candidate_uuid: "92ea0212286942cebd7b611fc3a9b816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c308465e7d1743d7b030159e95bd1c30" is_pre_election: true
I20260923 18:00:13.752997 18095 raft_consensus.cc:2454] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ea0212286942cebd7b611fc3a9b816 in term 0.
I20260923 18:00:13.753023 17813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7353b23384434019b3662ea98fa58dd7" candidate_uuid: "92ea0212286942cebd7b611fc3a9b816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" is_pre_election: true
I20260923 18:00:13.753289 17813 raft_consensus.cc:2454] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ea0212286942cebd7b611fc3a9b816 in term 0.
I20260923 18:00:13.753437 17889 leader_election.cc:302] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92ea0212286942cebd7b611fc3a9b816, c308465e7d1743d7b030159e95bd1c30; no voters:
I20260923 18:00:13.753758 18190 raft_consensus.cc:2781] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:13.753830 18190 raft_consensus.cc:482] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:13.753870 18190 raft_consensus.cc:3037] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:13.754851 18190 raft_consensus.cc:504] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.755213 18190 leader_election.cc:288] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [CANDIDATE]: Term 1 election: Requested vote from peers b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129:35339), c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297)
I20260923 18:00:13.755632 17813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7353b23384434019b3662ea98fa58dd7" candidate_uuid: "92ea0212286942cebd7b611fc3a9b816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00"
I20260923 18:00:13.755679 18095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7353b23384434019b3662ea98fa58dd7" candidate_uuid: "92ea0212286942cebd7b611fc3a9b816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c308465e7d1743d7b030159e95bd1c30"
I20260923 18:00:13.755761 17813 raft_consensus.cc:3037] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:13.755782 18095 raft_consensus.cc:3037] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:13.757004 18095 raft_consensus.cc:2454] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ea0212286942cebd7b611fc3a9b816 in term 1.
I20260923 18:00:13.757052 17813 raft_consensus.cc:2454] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ea0212286942cebd7b611fc3a9b816 in term 1.
I20260923 18:00:13.757301 17886 leader_election.cc:302] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92ea0212286942cebd7b611fc3a9b816, b21c3ea4f71a4e11ad05d23ad00efc00; no voters:
I20260923 18:00:13.757468 18190 raft_consensus.cc:2781] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:13.757788 18190 raft_consensus.cc:686] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [term 1 LEADER]: Becoming Leader. State: Replica: 92ea0212286942cebd7b611fc3a9b816, State: Running, Role: LEADER
I20260923 18:00:13.758184 18190 consensus_queue.cc:237] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.760726 18195 tablet_replica.cc:439] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: TxnStatusTablet state changed. Reason: New leader 92ea0212286942cebd7b611fc3a9b816. Latest consensus state: current_term: 1 leader_uuid: "92ea0212286942cebd7b611fc3a9b816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } } }
I20260923 18:00:13.760875 18195 tablet_replica.cc:442] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: This TxnStatusTablet replica's current role is: LEADER
I20260923 18:00:13.761616 18197 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 18:00:13.761837 18197 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 18:00:13.761994 17658 catalog_manager.cc:5495] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130). New cstate: current_term: 1 leader_uuid: "92ea0212286942cebd7b611fc3a9b816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260923 18:00:13.785825 18161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.781645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56356 (local address 127.15.139.130:42657)
0923 18:00:13.781824 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.781829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.781847 (+ 18us) server_negotiation.cc:407] Connection header received
0923 18:00:13.781937 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.781940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.782004 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.782120 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:13.782381 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.783179 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.784082 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.784440 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.784550 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.784857 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:13.785408 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:13.785520 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.785628 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.785687 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 18:00:13.785856 18182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.781437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:42657 (local address 127.0.0.1:56356)
0923 18:00:13.781642 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.781661 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.781789 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.782160 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.782162 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.782170 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 18:00:13.782266 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.782270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.783290 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.783293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.783966 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.783975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.784576 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.784579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.784670 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.784715 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 18:00:13.785514 (+ 799us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 18:00:13.785529 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.785593 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.785664 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260923 18:00:13.791486 17813 raft_consensus.cc:1261] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Refusing update from remote peer 92ea0212286942cebd7b611fc3a9b816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:13.791599 18095 raft_consensus.cc:1261] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Refusing update from remote peer 92ea0212286942cebd7b611fc3a9b816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:13.792228 18195 consensus_queue.cc:1035] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:13.792492 18190 consensus_queue.cc:1035] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:13.796254 18172 tablet_replica.cc:439] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00: TxnStatusTablet state changed. Reason: New leader 92ea0212286942cebd7b611fc3a9b816. Latest consensus state: current_term: 1 leader_uuid: "92ea0212286942cebd7b611fc3a9b816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } } }
I20260923 18:00:13.796391 18172 tablet_replica.cc:442] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:13.797371 18191 tablet_replica.cc:439] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30: TxnStatusTablet state changed. Reason: New leader 92ea0212286942cebd7b611fc3a9b816. Latest consensus state: current_term: 1 leader_uuid: "92ea0212286942cebd7b611fc3a9b816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } } }
I20260923 18:00:13.797490 18191 tablet_replica.cc:442] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:13.800181 18195 tablet_replica.cc:439] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92ea0212286942cebd7b611fc3a9b816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } } }
I20260923 18:00:13.800303 18195 tablet_replica.cc:442] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: This TxnStatusTablet replica's current role is: LEADER
I20260923 18:00:13.800482 18191 tablet_replica.cc:439] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92ea0212286942cebd7b611fc3a9b816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } } }
I20260923 18:00:13.800604 18191 tablet_replica.cc:442] T 7353b23384434019b3662ea98fa58dd7 P c308465e7d1743d7b030159e95bd1c30: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:13.809068 18190 tablet_replica.cc:439] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92ea0212286942cebd7b611fc3a9b816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } } }
I20260923 18:00:13.809226 18190 tablet_replica.cc:442] T 7353b23384434019b3662ea98fa58dd7 P 92ea0212286942cebd7b611fc3a9b816: This TxnStatusTablet replica's current role is: LEADER
I20260923 18:00:13.810269 18172 tablet_replica.cc:439] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92ea0212286942cebd7b611fc3a9b816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } } }
I20260923 18:00:13.810405 18172 tablet_replica.cc:442] T 7353b23384434019b3662ea98fa58dd7 P b21c3ea4f71a4e11ad05d23ad00efc00: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:13.815085 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.807430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.15.139.129:35339)
0923 18:00:13.807586 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.807591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.807770 (+ 179us) server_negotiation.cc:407] Connection header received
0923 18:00:13.807925 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.807927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.807980 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.808071 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:13.808413 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.809197 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.813270 (+ 4073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.813610 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.813684 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.814096 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:13.814607 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:13.814692 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.814887 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.814952 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 18:00:13.827685 18161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.823673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:42469 (local address 127.15.139.130:42657)
0923 18:00:13.823786 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.823791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.823940 (+ 149us) server_negotiation.cc:407] Connection header received
0923 18:00:13.824060 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.824063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.824114 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.824190 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:13.824503 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.825273 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.826538 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.826770 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.826853 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.827231 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:13.827301 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:13.827354 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.827485 (+ 131us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.827588 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 18:00:13.827762 18213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.823375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:42657 (local address 127.15.139.129:42469)
0923 18:00:13.823835 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.823849 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.823963 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.824226 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.824229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.824238 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 18:00:13.824393 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.824399 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.825403 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.825406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.826397 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.826408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.826892 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.826896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.827002 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.827067 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 18:00:13.827390 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 18:00:13.827396 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.827463 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.827577 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":234,"threads_started":1}
I20260923 18:00:13.835419 18095 raft_consensus.cc:1261] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Refusing update from remote peer b21c3ea4f71a4e11ad05d23ad00efc00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:13.835780 17955 raft_consensus.cc:1261] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Refusing update from remote peer b21c3ea4f71a4e11ad05d23ad00efc00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:13.835940 18175 consensus_queue.cc:1035] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:13.836297 18172 consensus_queue.cc:1035] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:13.840459 18200 mvcc.cc:205] Tried to move back new op lower bound from 7332603551065677824 to 7332603549870170112. Current Snapshot: MvccSnapshot[applied={T|T < 7332603551065677824}]
I20260923 18:00:13.840840 18201 mvcc.cc:205] Tried to move back new op lower bound from 7332603551065677824 to 7332603549870170112. Current Snapshot: MvccSnapshot[applied={T|T < 7332603551065677824}]
I20260923 18:00:13.846063 18202 mvcc.cc:205] Tried to move back new op lower bound from 7332603551065677824 to 7332603549870170112. Current Snapshot: MvccSnapshot[applied={T|T < 7332603551065677824}]
I20260923 18:00:13.858974 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.854828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:39663 (local address 127.15.139.129:35339)
0923 18:00:13.854961 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:13.854965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:13.855152 (+ 187us) server_negotiation.cc:407] Connection header received
0923 18:00:13.855310 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:13.855313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:13.855366 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:13.855441 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:13.855795 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.856591 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.857846 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.858111 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.858195 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.858523 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:13.858592 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:13.858641 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:13.858756 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 18:00:13.858815 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 18:00:13.859014 18217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:13.854746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35339 (local address 127.15.139.130:39663)
0923 18:00:13.855071 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:13.855083 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:13.855201 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:13.855463 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:13.855466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:13.855474 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 18:00:13.855655 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.855661 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.856726 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.856730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.857726 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:13.857737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:13.858223 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:13.858227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:13.858332 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:13.858389 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 18:00:13.858671 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 18:00:13.858677 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 18:00:13.858744 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 18:00:13.858836 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":105,"threads_started":1}
I20260923 18:00:13.887406 17653 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:13.887670 17653 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:13.890523 17653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99: Sending DeleteTablet for 3 replicas of tablet da2c773ca4584e4ab58997f0d9577e43
I20260923 18:00:13.891237 17795 tablet_service.cc:1514] Processing DeleteTablet for tablet da2c773ca4584e4ab58997f0d9577e43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:13 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:55024
I20260923 18:00:13.891381 18075 tablet_service.cc:1514] Processing DeleteTablet for tablet da2c773ca4584e4ab58997f0d9577e43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:13 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:47678
I20260923 18:00:13.891912 18220 tablet_replica.cc:330] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00: stopping tablet replica
I20260923 18:00:13.892170 17935 tablet_service.cc:1514] Processing DeleteTablet for tablet da2c773ca4584e4ab58997f0d9577e43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:13 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:56344
I20260923 18:00:13.892267 18220 raft_consensus.cc:2229] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:13.892730 18220 raft_consensus.cc:2258] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:13.893566 17653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:00:13.893997 17653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:13.894393 18220 ts_tablet_manager.cc:1905] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:13.897660 18221 tablet_replica.cc:330] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30: stopping tablet replica
I20260923 18:00:13.898017 18221 raft_consensus.cc:2229] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:13.898252 18221 raft_consensus.cc:2258] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:13.898650 18222 tablet_replica.cc:330] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816: stopping tablet replica
I20260923 18:00:13.899186 18222 raft_consensus.cc:2229] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:13.899346 18222 raft_consensus.cc:2258] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:13.900254 17795 tablet_service.cc:1467] Processing CreateTablet for tablet af13dfdb83404658812ca25d8fa5fd90 (DEFAULT_TABLE table=test-table [id=0c7b44f8160447d4bc14589e076052f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:13.900563 18221 ts_tablet_manager.cc:1905] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:13.900663 17935 tablet_service.cc:1467] Processing CreateTablet for tablet af13dfdb83404658812ca25d8fa5fd90 (DEFAULT_TABLE table=test-table [id=0c7b44f8160447d4bc14589e076052f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:13.900913 17935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af13dfdb83404658812ca25d8fa5fd90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:13.903270 18184 tablet_bootstrap.cc:492] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816: Bootstrap starting.
I20260923 18:00:13.904277 18184 tablet_bootstrap.cc:654] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:13.903465 18075 tablet_service.cc:1467] Processing CreateTablet for tablet af13dfdb83404658812ca25d8fa5fd90 (DEFAULT_TABLE table=test-table [id=0c7b44f8160447d4bc14589e076052f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:13.904640 18075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af13dfdb83404658812ca25d8fa5fd90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:13.905857 18222 ts_tablet_manager.cc:1905] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:13.906772 18185 tablet_bootstrap.cc:492] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30: Bootstrap starting.
I20260923 18:00:13.907243 17795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af13dfdb83404658812ca25d8fa5fd90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:13.907794 18185 tablet_bootstrap.cc:654] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:13.909520 18186 tablet_bootstrap.cc:492] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00: Bootstrap starting.
I20260923 18:00:13.910691 18222 ts_tablet_manager.cc:1918] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:13.910723 18186 tablet_bootstrap.cc:654] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:13.911394 18220 ts_tablet_manager.cc:1918] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:13.911586 18220 log.cc:1196] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/da2c773ca4584e4ab58997f0d9577e43
I20260923 18:00:13.911592 18222 log.cc:1196] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/da2c773ca4584e4ab58997f0d9577e43
I20260923 18:00:13.912022 18220 ts_tablet_manager.cc:1939] T da2c773ca4584e4ab58997f0d9577e43 P b21c3ea4f71a4e11ad05d23ad00efc00: Deleting consensus metadata
I20260923 18:00:13.912319 18222 ts_tablet_manager.cc:1939] T da2c773ca4584e4ab58997f0d9577e43 P 92ea0212286942cebd7b611fc3a9b816: Deleting consensus metadata
I20260923 18:00:13.913313 17642 catalog_manager.cc:4841] TS b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129:35339): tablet da2c773ca4584e4ab58997f0d9577e43 (table test-table [id=14c0ae98a6ae483c9a4a879644326207]) successfully deleted
I20260923 18:00:13.913973 18184 tablet_bootstrap.cc:492] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816: No bootstrap required, opened a new log
I20260923 18:00:13.914134 18184 ts_tablet_manager.cc:1397] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260923 18:00:13.914628 18184 raft_consensus.cc:348] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.914762 18184 raft_consensus.cc:374] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:13.914804 18184 raft_consensus.cc:729] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ea0212286942cebd7b611fc3a9b816, State: Initialized, Role: FOLLOWER
I20260923 18:00:13.914930 18184 consensus_queue.cc:260] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.915247 18184 ts_tablet_manager.cc:1428] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260923 18:00:13.911471 18221 ts_tablet_manager.cc:1918] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:13.915624 18221 log.cc:1196] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/da2c773ca4584e4ab58997f0d9577e43
I20260923 18:00:13.915969 18221 ts_tablet_manager.cc:1939] T da2c773ca4584e4ab58997f0d9577e43 P c308465e7d1743d7b030159e95bd1c30: Deleting consensus metadata
I20260923 18:00:13.917819 17644 catalog_manager.cc:4841] TS 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130:42657): tablet da2c773ca4584e4ab58997f0d9577e43 (table test-table [id=14c0ae98a6ae483c9a4a879644326207]) successfully deleted
I20260923 18:00:13.918500 18186 tablet_bootstrap.cc:492] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00: No bootstrap required, opened a new log
I20260923 18:00:13.918612 18186 ts_tablet_manager.cc:1397] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260923 18:00:13.918910 17645 catalog_manager.cc:4841] TS c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297): tablet da2c773ca4584e4ab58997f0d9577e43 (table test-table [id=14c0ae98a6ae483c9a4a879644326207]) successfully deleted
I20260923 18:00:13.919119 18186 raft_consensus.cc:348] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.919252 18186 raft_consensus.cc:374] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:13.919291 18186 raft_consensus.cc:729] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b21c3ea4f71a4e11ad05d23ad00efc00, State: Initialized, Role: FOLLOWER
I20260923 18:00:13.919427 18186 consensus_queue.cc:260] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.919770 18186 ts_tablet_manager.cc:1428] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260923 18:00:13.920362 18185 tablet_bootstrap.cc:492] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30: No bootstrap required, opened a new log
I20260923 18:00:13.920460 18185 ts_tablet_manager.cc:1397] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260923 18:00:13.920867 18185 raft_consensus.cc:348] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.920990 18185 raft_consensus.cc:374] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:13.921031 18185 raft_consensus.cc:729] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c308465e7d1743d7b030159e95bd1c30, State: Initialized, Role: FOLLOWER
I20260923 18:00:13.921172 18185 consensus_queue.cc:260] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:13.921442 18185 ts_tablet_manager.cc:1428] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:14.106923 18190 raft_consensus.cc:482] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:14.107087 18190 raft_consensus.cc:504] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:14.107652 18190 leader_election.cc:288] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129:35339), c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297)
I20260923 18:00:14.108168 18095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af13dfdb83404658812ca25d8fa5fd90" candidate_uuid: "92ea0212286942cebd7b611fc3a9b816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c308465e7d1743d7b030159e95bd1c30" is_pre_election: true
I20260923 18:00:14.108173 17813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af13dfdb83404658812ca25d8fa5fd90" candidate_uuid: "92ea0212286942cebd7b611fc3a9b816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" is_pre_election: true
I20260923 18:00:14.108346 18095 raft_consensus.cc:2454] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ea0212286942cebd7b611fc3a9b816 in term 0.
I20260923 18:00:14.108351 17813 raft_consensus.cc:2454] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ea0212286942cebd7b611fc3a9b816 in term 0.
I20260923 18:00:14.108705 17889 leader_election.cc:302] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92ea0212286942cebd7b611fc3a9b816, c308465e7d1743d7b030159e95bd1c30; no voters:
I20260923 18:00:14.108875 18190 raft_consensus.cc:2781] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:14.108947 18190 raft_consensus.cc:482] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:14.108986 18190 raft_consensus.cc:3037] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:14.109978 18190 raft_consensus.cc:504] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:14.110339 18190 leader_election.cc:288] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [CANDIDATE]: Term 1 election: Requested vote from peers b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129:35339), c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297)
I20260923 18:00:14.110710 17813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af13dfdb83404658812ca25d8fa5fd90" candidate_uuid: "92ea0212286942cebd7b611fc3a9b816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00"
I20260923 18:00:14.110711 18095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af13dfdb83404658812ca25d8fa5fd90" candidate_uuid: "92ea0212286942cebd7b611fc3a9b816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c308465e7d1743d7b030159e95bd1c30"
I20260923 18:00:14.110839 17813 raft_consensus.cc:3037] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:14.110839 18095 raft_consensus.cc:3037] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:14.112071 17813 raft_consensus.cc:2454] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ea0212286942cebd7b611fc3a9b816 in term 1.
I20260923 18:00:14.112090 18095 raft_consensus.cc:2454] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ea0212286942cebd7b611fc3a9b816 in term 1.
I20260923 18:00:14.112418 17889 leader_election.cc:302] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92ea0212286942cebd7b611fc3a9b816, c308465e7d1743d7b030159e95bd1c30; no voters:
I20260923 18:00:14.112586 18190 raft_consensus.cc:2781] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:14.112681 18190 raft_consensus.cc:686] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 1 LEADER]: Becoming Leader. State: Replica: 92ea0212286942cebd7b611fc3a9b816, State: Running, Role: LEADER
I20260923 18:00:14.112840 18190 consensus_queue.cc:237] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:14.114576 17653 catalog_manager.cc:5495] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130). New cstate: current_term: 1 leader_uuid: "92ea0212286942cebd7b611fc3a9b816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260923 18:00:14.153111 18161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:14.150081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56372 (local address 127.15.139.130:42657)
0923 18:00:14.150192 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:14.150195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:14.150212 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:00:14.150358 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:14.150360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:14.150401 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:14.150465 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:14.150710 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:14.151256 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:14.152144 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:14.152364 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:14.152459 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:14.152669 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:14.152738 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:14.152792 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:14.152872 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 18:00:14.152919 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 18:00:14.162282 18217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:14.158870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:42657 (local address 127.15.139.130:37339)
0923 18:00:14.159064 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:14.159079 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:14.159240 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:14.159529 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:14.159533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:14.159539 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 18:00:14.159621 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:14.159628 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:14.160518 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:14.160520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:14.161176 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:14.161186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:14.161616 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:14.161618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:14.161675 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:14.161699 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 18:00:14.161972 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 18:00:14.162030 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 18:00:14.162088 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 18:00:14.162154 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260923 18:00:14.162282 18161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:14.158983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:37339 (local address 127.15.139.130:42657)
0923 18:00:14.159078 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:14.159083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:14.159147 (+ 64us) server_negotiation.cc:407] Connection header received
0923 18:00:14.159360 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:14.159364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:14.159410 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:14.159488 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:14.159727 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:14.160374 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:14.161274 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:14.161478 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:14.161611 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:14.161799 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:14.161878 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:14.161935 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:14.162115 (+ 180us) server_negotiation.cc:299] Negotiation successful
0923 18:00:14.162158 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 18:00:14.167773 18095 raft_consensus.cc:1261] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Refusing update from remote peer 92ea0212286942cebd7b611fc3a9b816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:14.167830 17813 raft_consensus.cc:1261] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Refusing update from remote peer 92ea0212286942cebd7b611fc3a9b816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:14.168210 18209 consensus_queue.cc:1035] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:14.168357 18190 consensus_queue.cc:1035] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 18:00:14.181648 18200 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 18:00:15.747445 17657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:15.747680 17657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:15.750440 17657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99: Sending DeleteTablet for 3 replicas of tablet af13dfdb83404658812ca25d8fa5fd90
I20260923 18:00:15.751087 17795 tablet_service.cc:1514] Processing DeleteTablet for tablet af13dfdb83404658812ca25d8fa5fd90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:15 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:55024
I20260923 18:00:15.751204 17934 tablet_service.cc:1514] Processing DeleteTablet for tablet af13dfdb83404658812ca25d8fa5fd90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:15 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:56344
I20260923 18:00:15.751212 18075 tablet_service.cc:1514] Processing DeleteTablet for tablet af13dfdb83404658812ca25d8fa5fd90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:15 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:47678
I20260923 18:00:15.751580 18254 tablet_replica.cc:330] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816: stopping tablet replica
I20260923 18:00:15.751590 18252 tablet_replica.cc:330] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00: stopping tablet replica
I20260923 18:00:15.751748 18252 raft_consensus.cc:2229] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:15.751763 18253 tablet_replica.cc:330] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30: stopping tablet replica
I20260923 18:00:15.751781 18254 raft_consensus.cc:2229] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:15.751883 18253 raft_consensus.cc:2229] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:15.751968 18252 raft_consensus.cc:2258] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:15.752041 18253 raft_consensus.cc:2258] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:15.752116 18254 raft_consensus.cc:2258] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:15.752947 18254 ts_tablet_manager.cc:1905] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:15.754199 18252 ts_tablet_manager.cc:1905] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:15.755221 18253 ts_tablet_manager.cc:1905] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:15.756038 18254 ts_tablet_manager.cc:1918] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 18:00:15.756176 18254 log.cc:1196] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/af13dfdb83404658812ca25d8fa5fd90
I20260923 18:00:15.756559 18254 ts_tablet_manager.cc:1939] T af13dfdb83404658812ca25d8fa5fd90 P 92ea0212286942cebd7b611fc3a9b816: Deleting consensus metadata
I20260923 18:00:15.757076 18252 ts_tablet_manager.cc:1918] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 18:00:15.757218 18252 log.cc:1196] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/af13dfdb83404658812ca25d8fa5fd90
I20260923 18:00:15.757372 17644 catalog_manager.cc:4841] TS 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130:42657): tablet af13dfdb83404658812ca25d8fa5fd90 (table test-table [id=0c7b44f8160447d4bc14589e076052f8]) successfully deleted
I20260923 18:00:15.758003 18253 ts_tablet_manager.cc:1918] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 18:00:15.758096 18253 log.cc:1196] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/af13dfdb83404658812ca25d8fa5fd90
I20260923 18:00:15.758157 18252 ts_tablet_manager.cc:1939] T af13dfdb83404658812ca25d8fa5fd90 P b21c3ea4f71a4e11ad05d23ad00efc00: Deleting consensus metadata
I20260923 18:00:15.758402 18253 ts_tablet_manager.cc:1939] T af13dfdb83404658812ca25d8fa5fd90 P c308465e7d1743d7b030159e95bd1c30: Deleting consensus metadata
I20260923 18:00:15.758862 17642 catalog_manager.cc:4841] TS b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129:35339): tablet af13dfdb83404658812ca25d8fa5fd90 (table test-table [id=0c7b44f8160447d4bc14589e076052f8]) successfully deleted
I20260923 18:00:15.759054 17645 catalog_manager.cc:4841] TS c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297): tablet af13dfdb83404658812ca25d8fa5fd90 (table test-table [id=0c7b44f8160447d4bc14589e076052f8]) successfully deleted
Sep 23 18:00:15 dist-test-slave-vr57 krb5kdc[17613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 18:00:15 dist-test-slave-vr57 krb5kdc[17613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:15.787041 18262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:15.771267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49768 (local address 127.15.139.190:40067)
0923 18:00:15.771602 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:15.771612 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:15.771743 (+ 131us) server_negotiation.cc:407] Connection header received
0923 18:00:15.771926 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:15.771929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:15.771982 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:15.772139 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:15.772856 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:15.773447 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:15.774347 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:15.774592 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:15.774705 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:15.783141 (+ 8436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:15.783145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:15.783147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:15.783177 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:15.784991 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:15.785571 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:15.785575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:15.785580 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:15.785681 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:15.786009 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:15.786012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:15.786014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:15.786232 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:15.786403 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:15.786686 (+ 283us) server_negotiation.cc:299] Negotiation successful
0923 18:00:15.786829 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":143,"threads_started":1}
I20260923 18:00:15.790014 17657 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:00:15.790601 17657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:15.795710 17795 tablet_service.cc:1467] Processing CreateTablet for tablet b84a659cbc154f259568ffd83b731a12 (DEFAULT_TABLE table=test-table [id=ca7b2dd7cc1e4e6e9e72bacd3a1230ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:15.795856 17934 tablet_service.cc:1467] Processing CreateTablet for tablet b84a659cbc154f259568ffd83b731a12 (DEFAULT_TABLE table=test-table [id=ca7b2dd7cc1e4e6e9e72bacd3a1230ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:15.795873 18075 tablet_service.cc:1467] Processing CreateTablet for tablet b84a659cbc154f259568ffd83b731a12 (DEFAULT_TABLE table=test-table [id=ca7b2dd7cc1e4e6e9e72bacd3a1230ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:15.796027 17795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84a659cbc154f259568ffd83b731a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:15.796130 18075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84a659cbc154f259568ffd83b731a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:15.796151 17934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84a659cbc154f259568ffd83b731a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:15.798619 18263 tablet_bootstrap.cc:492] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30: Bootstrap starting.
I20260923 18:00:15.799160 18265 tablet_bootstrap.cc:492] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816: Bootstrap starting.
I20260923 18:00:15.799677 18263 tablet_bootstrap.cc:654] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:15.799690 18264 tablet_bootstrap.cc:492] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00: Bootstrap starting.
I20260923 18:00:15.800238 18265 tablet_bootstrap.cc:654] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:15.800452 18264 tablet_bootstrap.cc:654] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:15.801090 18263 tablet_bootstrap.cc:492] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30: No bootstrap required, opened a new log
I20260923 18:00:15.801196 18263 ts_tablet_manager.cc:1397] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:15.801533 18265 tablet_bootstrap.cc:492] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816: No bootstrap required, opened a new log
I20260923 18:00:15.801608 18265 ts_tablet_manager.cc:1397] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:15.801713 18264 tablet_bootstrap.cc:492] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00: No bootstrap required, opened a new log
I20260923 18:00:15.801698 18263 raft_consensus.cc:348] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.801806 18263 raft_consensus.cc:374] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:15.801806 18264 ts_tablet_manager.cc:1397] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:15.801841 18263 raft_consensus.cc:729] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c308465e7d1743d7b030159e95bd1c30, State: Initialized, Role: FOLLOWER
I20260923 18:00:15.801961 18263 consensus_queue.cc:260] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.802042 18265 raft_consensus.cc:348] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.802147 18265 raft_consensus.cc:374] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:15.802181 18265 raft_consensus.cc:729] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ea0212286942cebd7b611fc3a9b816, State: Initialized, Role: FOLLOWER
I20260923 18:00:15.802342 18263 ts_tablet_manager.cc:1428] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:15.802304 18265 consensus_queue.cc:260] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.802292 18264 raft_consensus.cc:348] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.802407 18264 raft_consensus.cc:374] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:15.802466 18264 raft_consensus.cc:729] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b21c3ea4f71a4e11ad05d23ad00efc00, State: Initialized, Role: FOLLOWER
I20260923 18:00:15.802593 18265 ts_tablet_manager.cc:1428] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260923 18:00:15.802634 18264 consensus_queue.cc:260] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.803073 18264 ts_tablet_manager.cc:1428] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260923 18:00:15.894357 18270 raft_consensus.cc:482] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:15.894564 18270 raft_consensus.cc:504] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.894981 18270 leader_election.cc:288] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130:42657), c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297)
I20260923 18:00:15.895608 18094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84a659cbc154f259568ffd83b731a12" candidate_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c308465e7d1743d7b030159e95bd1c30" is_pre_election: true
I20260923 18:00:15.895605 17955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84a659cbc154f259568ffd83b731a12" candidate_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ea0212286942cebd7b611fc3a9b816" is_pre_election: true
I20260923 18:00:15.895747 18094 raft_consensus.cc:2454] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b21c3ea4f71a4e11ad05d23ad00efc00 in term 0.
I20260923 18:00:15.895780 17955 raft_consensus.cc:2454] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b21c3ea4f71a4e11ad05d23ad00efc00 in term 0.
I20260923 18:00:15.896212 17748 leader_election.cc:302] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92ea0212286942cebd7b611fc3a9b816, b21c3ea4f71a4e11ad05d23ad00efc00; no voters:
I20260923 18:00:15.896435 18270 raft_consensus.cc:2781] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:15.896543 18270 raft_consensus.cc:482] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:15.896579 18270 raft_consensus.cc:3037] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:15.897823 18270 raft_consensus.cc:504] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.898211 18270 leader_election.cc:288] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [CANDIDATE]: Term 1 election: Requested vote from peers 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130:42657), c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297)
I20260923 18:00:15.898633 17955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84a659cbc154f259568ffd83b731a12" candidate_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ea0212286942cebd7b611fc3a9b816"
I20260923 18:00:15.898770 17955 raft_consensus.cc:3037] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:15.898932 18094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84a659cbc154f259568ffd83b731a12" candidate_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c308465e7d1743d7b030159e95bd1c30"
I20260923 18:00:15.899053 18094 raft_consensus.cc:3037] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:15.899858 17955 raft_consensus.cc:2454] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b21c3ea4f71a4e11ad05d23ad00efc00 in term 1.
I20260923 18:00:15.900144 17748 leader_election.cc:302] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92ea0212286942cebd7b611fc3a9b816, b21c3ea4f71a4e11ad05d23ad00efc00; no voters:
I20260923 18:00:15.900190 18094 raft_consensus.cc:2454] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b21c3ea4f71a4e11ad05d23ad00efc00 in term 1.
I20260923 18:00:15.900321 18270 raft_consensus.cc:2781] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:15.900557 18270 raft_consensus.cc:686] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 LEADER]: Becoming Leader. State: Replica: b21c3ea4f71a4e11ad05d23ad00efc00, State: Running, Role: LEADER
I20260923 18:00:15.900715 18270 consensus_queue.cc:237] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.902491 17658 catalog_manager.cc:5495] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 reported cstate change: term changed from 0 to 1, leader changed from <none> to b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129). New cstate: current_term: 1 leader_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260923 18:00:15.935284 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:15.932030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55042 (local address 127.15.139.129:35339)
0923 18:00:15.932316 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:15.932323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:15.932336 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:00:15.932377 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:15.932379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:15.932455 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:15.932534 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:15.932774 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:15.933339 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:15.934209 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:15.934457 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:15.934548 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:15.934794 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:15.934865 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:15.934937 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:15.935029 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 18:00:15.935078 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":129,"threads_started":1}
I20260923 18:00:15.942927 17955 raft_consensus.cc:1261] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Refusing update from remote peer b21c3ea4f71a4e11ad05d23ad00efc00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:15.942919 18094 raft_consensus.cc:1261] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Refusing update from remote peer b21c3ea4f71a4e11ad05d23ad00efc00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:15.943363 18271 consensus_queue.cc:1035] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:15.943568 18270 consensus_queue.cc:1035] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:15.977393 17658 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:15.977555 17658 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:15.979863 17658 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99: Sending DeleteTablet for 3 replicas of tablet b84a659cbc154f259568ffd83b731a12
I20260923 18:00:15.980556 17934 tablet_service.cc:1514] Processing DeleteTablet for tablet b84a659cbc154f259568ffd83b731a12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:15 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:56344
I20260923 18:00:15.980650 17795 tablet_service.cc:1514] Processing DeleteTablet for tablet b84a659cbc154f259568ffd83b731a12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:15 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:55024
I20260923 18:00:15.980758 18254 tablet_replica.cc:330] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816: stopping tablet replica
I20260923 18:00:15.980789 18075 tablet_service.cc:1514] Processing DeleteTablet for tablet b84a659cbc154f259568ffd83b731a12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:15 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:47678
I20260923 18:00:15.980868 18254 raft_consensus.cc:2229] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:15.980895 18252 tablet_replica.cc:330] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00: stopping tablet replica
I20260923 18:00:15.980988 18254 raft_consensus.cc:2258] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:15.981027 18252 raft_consensus.cc:2229] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:15.981259 18252 raft_consensus.cc:2258] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:15.981525 18253 tablet_replica.cc:330] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30: stopping tablet replica
I20260923 18:00:15.981490 17658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 18:00:15.981662 18253 raft_consensus.cc:2229] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:15.981772 18253 raft_consensus.cc:2258] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 18:00:15.981966 17658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:15.982003 18254 ts_tablet_manager.cc:1905] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:15.982421 18252 ts_tablet_manager.cc:1905] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:15.982733 18253 ts_tablet_manager.cc:1905] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:15.984758 18254 ts_tablet_manager.cc:1918] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:15.984831 18254 log.cc:1196] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/b84a659cbc154f259568ffd83b731a12
I20260923 18:00:15.984930 18253 ts_tablet_manager.cc:1918] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:15.984987 18253 log.cc:1196] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/b84a659cbc154f259568ffd83b731a12
I20260923 18:00:15.985102 18254 ts_tablet_manager.cc:1939] T b84a659cbc154f259568ffd83b731a12 P 92ea0212286942cebd7b611fc3a9b816: Deleting consensus metadata
I20260923 18:00:15.985157 18252 ts_tablet_manager.cc:1918] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:15.985236 18252 log.cc:1196] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/b84a659cbc154f259568ffd83b731a12
I20260923 18:00:15.985296 18253 ts_tablet_manager.cc:1939] T b84a659cbc154f259568ffd83b731a12 P c308465e7d1743d7b030159e95bd1c30: Deleting consensus metadata
I20260923 18:00:15.985483 18252 ts_tablet_manager.cc:1939] T b84a659cbc154f259568ffd83b731a12 P b21c3ea4f71a4e11ad05d23ad00efc00: Deleting consensus metadata
I20260923 18:00:15.985973 17644 catalog_manager.cc:4841] TS 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130:42657): tablet b84a659cbc154f259568ffd83b731a12 (table test-table [id=ca7b2dd7cc1e4e6e9e72bacd3a1230ae]) successfully deleted
I20260923 18:00:15.986011 17645 catalog_manager.cc:4841] TS c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297): tablet b84a659cbc154f259568ffd83b731a12 (table test-table [id=ca7b2dd7cc1e4e6e9e72bacd3a1230ae]) successfully deleted
I20260923 18:00:15.986344 17642 catalog_manager.cc:4841] TS b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129:35339): tablet b84a659cbc154f259568ffd83b731a12 (table test-table [id=ca7b2dd7cc1e4e6e9e72bacd3a1230ae]) successfully deleted
I20260923 18:00:15.989521 17795 tablet_service.cc:1467] Processing CreateTablet for tablet 95391b9926344f909af52a15cb16e6f7 (DEFAULT_TABLE table=test-table [id=e16fb4f6b000487ea50b128d58377b32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:15.989521 17934 tablet_service.cc:1467] Processing CreateTablet for tablet 95391b9926344f909af52a15cb16e6f7 (DEFAULT_TABLE table=test-table [id=e16fb4f6b000487ea50b128d58377b32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:15.989645 18075 tablet_service.cc:1467] Processing CreateTablet for tablet 95391b9926344f909af52a15cb16e6f7 (DEFAULT_TABLE table=test-table [id=e16fb4f6b000487ea50b128d58377b32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:15.989769 17795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95391b9926344f909af52a15cb16e6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:15.989769 17934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95391b9926344f909af52a15cb16e6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:15.989871 18075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95391b9926344f909af52a15cb16e6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:15.991919 18263 tablet_bootstrap.cc:492] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30: Bootstrap starting.
I20260923 18:00:15.991943 18264 tablet_bootstrap.cc:492] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00: Bootstrap starting.
I20260923 18:00:15.992615 18264 tablet_bootstrap.cc:654] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:15.992734 18263 tablet_bootstrap.cc:654] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:15.993362 18265 tablet_bootstrap.cc:492] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816: Bootstrap starting.
I20260923 18:00:15.993815 18264 tablet_bootstrap.cc:492] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00: No bootstrap required, opened a new log
I20260923 18:00:15.993919 18264 ts_tablet_manager.cc:1397] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:15.994251 18265 tablet_bootstrap.cc:654] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:15.994272 18263 tablet_bootstrap.cc:492] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30: No bootstrap required, opened a new log
I20260923 18:00:15.994350 18263 ts_tablet_manager.cc:1397] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:15.994326 18264 raft_consensus.cc:348] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.994451 18264 raft_consensus.cc:374] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:15.994491 18264 raft_consensus.cc:729] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b21c3ea4f71a4e11ad05d23ad00efc00, State: Initialized, Role: FOLLOWER
I20260923 18:00:15.994616 18264 consensus_queue.cc:260] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.994786 18263 raft_consensus.cc:348] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.994874 18264 ts_tablet_manager.cc:1428] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260923 18:00:15.994881 18263 raft_consensus.cc:374] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:15.994931 18263 raft_consensus.cc:729] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c308465e7d1743d7b030159e95bd1c30, State: Initialized, Role: FOLLOWER
I20260923 18:00:15.995061 18263 consensus_queue.cc:260] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.995322 18263 ts_tablet_manager.cc:1428] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:15.995597 18265 tablet_bootstrap.cc:492] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816: No bootstrap required, opened a new log
I20260923 18:00:15.995697 18265 ts_tablet_manager.cc:1397] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:15.996176 18265 raft_consensus.cc:348] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.996304 18265 raft_consensus.cc:374] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:15.996366 18265 raft_consensus.cc:729] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ea0212286942cebd7b611fc3a9b816, State: Initialized, Role: FOLLOWER
I20260923 18:00:15.996543 18265 consensus_queue.cc:260] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:15.996831 18265 ts_tablet_manager.cc:1428] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:16.207219 18269 raft_consensus.cc:482] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:16.207391 18269 raft_consensus.cc:504] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:16.208638 18269 leader_election.cc:288] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129:35339), 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130:42657)
I20260923 18:00:16.209504 18239 raft_consensus.cc:482] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:16.209635 18239 raft_consensus.cc:504] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:16.210384 18239 leader_election.cc:288] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129:35339), c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297)
I20260923 18:00:16.210928 18095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95391b9926344f909af52a15cb16e6f7" candidate_uuid: "92ea0212286942cebd7b611fc3a9b816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c308465e7d1743d7b030159e95bd1c30" is_pre_election: true
I20260923 18:00:16.210953 17813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95391b9926344f909af52a15cb16e6f7" candidate_uuid: "92ea0212286942cebd7b611fc3a9b816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" is_pre_election: true
I20260923 18:00:16.211078 18095 raft_consensus.cc:2454] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ea0212286942cebd7b611fc3a9b816 in term 0.
I20260923 18:00:16.211110 17813 raft_consensus.cc:2454] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ea0212286942cebd7b611fc3a9b816 in term 0.
I20260923 18:00:16.211472 17886 leader_election.cc:302] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92ea0212286942cebd7b611fc3a9b816, b21c3ea4f71a4e11ad05d23ad00efc00; no voters:
I20260923 18:00:16.211740 18239 raft_consensus.cc:2781] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:16.211808 18239 raft_consensus.cc:482] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:16.211843 18239 raft_consensus.cc:3037] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:16.213007 18239 raft_consensus.cc:504] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:16.213397 18239 leader_election.cc:288] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [CANDIDATE]: Term 1 election: Requested vote from peers b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129:35339), c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297)
I20260923 18:00:16.213766 17813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95391b9926344f909af52a15cb16e6f7" candidate_uuid: "92ea0212286942cebd7b611fc3a9b816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00"
I20260923 18:00:16.213912 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:16.208909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:42657 (local address 127.15.139.131:48989)
0923 18:00:16.209292 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:16.209307 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:16.209460 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:16.209890 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:16.209892 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:16.209908 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:16.210041 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:16.210047 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:16.211095 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:16.211098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:16.212385 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:16.212395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:16.213375 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:16.213378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:16.213542 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:16.213573 (+ 31us) client_negotiation.cc:769] Sending connection context
0923 18:00:16.213651 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 18:00:16.213731 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":92,"threads_started":1}
I20260923 18:00:16.213919 17813 raft_consensus.cc:3037] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:16.214226 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:16.209119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:58045 (local address 127.15.139.129:35339)
0923 18:00:16.209257 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:16.209262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:16.209543 (+ 281us) server_negotiation.cc:407] Connection header received
0923 18:00:16.209740 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:16.209745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:16.209795 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:16.209864 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:16.210331 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:16.211025 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:16.212468 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:16.213105 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:16.213244 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:16.213305 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:16.213777 (+ 472us) server_negotiation.cc:299] Negotiation successful
0923 18:00:16.213856 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 18:00:16.214304 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:16.208952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35339 (local address 127.15.139.131:58045)
0923 18:00:16.209423 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:16.209441 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:16.209572 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:16.209940 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:16.209943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:16.209964 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:16.210103 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:16.210110 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:16.211204 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:16.211207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:16.212329 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:16.212346 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:16.213374 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:16.213378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:16.213653 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:16.213678 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 18:00:16.213761 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 18:00:16.213829 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":140,"thread_start_us":112,"threads_started":1}
I20260923 18:00:16.214318 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:16.209065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:48989 (local address 127.15.139.130:42657)
0923 18:00:16.209380 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:16.209398 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:16.209421 (+ 23us) server_negotiation.cc:407] Connection header received
0923 18:00:16.209626 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:16.209630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:16.209683 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:16.209758 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:16.210195 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:16.210963 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:16.212524 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:16.213146 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:16.213261 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:16.213312 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:16.214079 (+ 767us) server_negotiation.cc:299] Negotiation successful
0923 18:00:16.214185 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":92,"threads_started":1}
I20260923 18:00:16.214630 18095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95391b9926344f909af52a15cb16e6f7" candidate_uuid: "92ea0212286942cebd7b611fc3a9b816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c308465e7d1743d7b030159e95bd1c30"
I20260923 18:00:16.214752 18095 raft_consensus.cc:3037] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:16.214731 17955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95391b9926344f909af52a15cb16e6f7" candidate_uuid: "c308465e7d1743d7b030159e95bd1c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ea0212286942cebd7b611fc3a9b816" is_pre_election: true
I20260923 18:00:16.214924 17955 raft_consensus.cc:2379] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c308465e7d1743d7b030159e95bd1c30 in current term 1: Already voted for candidate 92ea0212286942cebd7b611fc3a9b816 in this term.
I20260923 18:00:16.215291 17813 raft_consensus.cc:2454] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ea0212286942cebd7b611fc3a9b816 in term 1.
I20260923 18:00:16.215454 17814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95391b9926344f909af52a15cb16e6f7" candidate_uuid: "c308465e7d1743d7b030159e95bd1c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" is_pre_election: true
I20260923 18:00:16.215670 17814 raft_consensus.cc:2379] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c308465e7d1743d7b030159e95bd1c30 in current term 1: Already voted for candidate 92ea0212286942cebd7b611fc3a9b816 in this term.
I20260923 18:00:16.215670 17886 leader_election.cc:302] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92ea0212286942cebd7b611fc3a9b816, b21c3ea4f71a4e11ad05d23ad00efc00; no voters:
I20260923 18:00:16.215886 18239 raft_consensus.cc:2781] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:16.215979 18239 raft_consensus.cc:686] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 1 LEADER]: Becoming Leader. State: Replica: 92ea0212286942cebd7b611fc3a9b816, State: Running, Role: LEADER
I20260923 18:00:16.216101 18095 raft_consensus.cc:2454] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ea0212286942cebd7b611fc3a9b816 in term 1.
I20260923 18:00:16.216128 18239 consensus_queue.cc:237] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } }
I20260923 18:00:16.216050 18026 leader_election.cc:302] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c308465e7d1743d7b030159e95bd1c30; no voters: 92ea0212286942cebd7b611fc3a9b816, b21c3ea4f71a4e11ad05d23ad00efc00
I20260923 18:00:16.216419 18269 raft_consensus.cc:2726] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 18:00:16.218187 17658 catalog_manager.cc:5495] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130). New cstate: current_term: 1 leader_uuid: "92ea0212286942cebd7b611fc3a9b816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92ea0212286942cebd7b611fc3a9b816" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260923 18:00:16.244287 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:16.241331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.15.139.130:42657)
0923 18:00:16.241458 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:16.241462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:16.241475 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:00:16.241564 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:16.241566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:16.241608 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:16.241672 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:16.241884 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:16.242413 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:16.243286 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:16.243492 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:16.243640 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:16.243841 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:16.243911 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:16.243963 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:16.244066 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 18:00:16.244112 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 18:00:16.250590 18095 raft_consensus.cc:1261] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Refusing update from remote peer 92ea0212286942cebd7b611fc3a9b816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:16.250813 17814 raft_consensus.cc:1261] T 95391b9926344f909af52a15cb16e6f7 P b21c3ea4f71a4e11ad05d23ad00efc00 [term 1 FOLLOWER]: Refusing update from remote peer 92ea0212286942cebd7b611fc3a9b816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:16.250977 18231 consensus_queue.cc:1035] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c308465e7d1743d7b030159e95bd1c30" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:16.251184 18239 consensus_queue.cc:1035] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b21c3ea4f71a4e11ad05d23ad00efc00" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 18:00:16.264074 18232 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 18:00:17.825181 17653 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:17.825353 17653 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:17.828205 17653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 535877aafb4a49688ef43c18236f5e99: Sending DeleteTablet for 3 replicas of tablet 95391b9926344f909af52a15cb16e6f7
I20260923 18:00:17.828799 17934 tablet_service.cc:1514] Processing DeleteTablet for tablet 95391b9926344f909af52a15cb16e6f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:17 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:56344
I20260923 18:00:17.828830 17795 tablet_service.cc:1514] Processing DeleteTablet for tablet 95391b9926344f909af52a15cb16e6f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:17 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:55024
I20260923 18:00:17.829157 18075 tablet_service.cc:1514] Processing DeleteTablet for tablet 95391b9926344f909af52a15cb16e6f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:17 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:47678
I20260923 18:00:17.829404 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 17730
I20260923 18:00:17.831141 18308 tablet_replica.cc:330] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816: stopping tablet replica
I20260923 18:00:17.831300 18308 raft_consensus.cc:2229] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:17.831522 18308 raft_consensus.cc:2258] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:17.833309 18310 tablet_replica.cc:330] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30: stopping tablet replica
I20260923 18:00:17.833475 18310 raft_consensus.cc:2229] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:17.833606 18310 raft_consensus.cc:2258] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:17.834342 18310 ts_tablet_manager.cc:1905] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:17.836262 18308 ts_tablet_manager.cc:1905] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:17.837402 18310 ts_tablet_manager.cc:1918] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 18:00:17.837492 18310 log.cc:1196] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/95391b9926344f909af52a15cb16e6f7
I20260923 18:00:17.837767 18310 ts_tablet_manager.cc:1939] T 95391b9926344f909af52a15cb16e6f7 P c308465e7d1743d7b030159e95bd1c30: Deleting consensus metadata
I20260923 18:00:17.838703 17645 catalog_manager.cc:4841] TS c308465e7d1743d7b030159e95bd1c30 (127.15.139.131:42297): tablet 95391b9926344f909af52a15cb16e6f7 (table test-table [id=e16fb4f6b000487ea50b128d58377b32]) successfully deleted
I20260923 18:00:17.839032 18308 ts_tablet_manager.cc:1918] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 18:00:17.839149 18308 log.cc:1196] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/95391b9926344f909af52a15cb16e6f7
I20260923 18:00:17.839423 18308 ts_tablet_manager.cc:1939] T 95391b9926344f909af52a15cb16e6f7 P 92ea0212286942cebd7b611fc3a9b816: Deleting consensus metadata
I20260923 18:00:17.841516 17644 catalog_manager.cc:4841] TS 92ea0212286942cebd7b611fc3a9b816 (127.15.139.130:42657): tablet 95391b9926344f909af52a15cb16e6f7 (table test-table [id=e16fb4f6b000487ea50b128d58377b32]) successfully deleted
I20260923 18:00:17.843281 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 17870
W20260923 18:00:17.843554 17642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:35339 (error 108)
I20260923 18:00:17.845094 18312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:17.844574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35339 (local address 127.0.0.1:45988)
0923 18:00:17.844866 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:17.844960 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:35339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":86,"threads_started":1}
W20260923 18:00:17.845219 17642 catalog_manager.cc:4584] TS b21c3ea4f71a4e11ad05d23ad00efc00 (127.15.139.129:35339): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95391b9926344f909af52a15cb16e6f7: Network error: Client connection negotiation failed: client connection to 127.15.139.129:35339: connect: Connection refused (error 111)
I20260923 18:00:17.851361 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 18010
I20260923 18:00:17.857366 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 17626
2026-09-23T18:00:17Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6933 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:17 dist-test-slave-vr57 krb5kdc[18315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:17 dist-test-slave-vr57 krb5kdc[18315](info): set up 2 sockets
Sep 23 18:00:17 dist-test-slave-vr57 krb5kdc[18315](info): commencing operation
krb5kdc: starting...
W20260923 18:00:18.908186 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:18 dist-test-slave-vr57 krb5kdc[18315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:00:19.038966 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40509
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:40509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:19.152259 18328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:19.152616 18328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:19.152729 18328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:19.157387 18328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:19.157486 18328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:19.157531 18328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:19.157595 18328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:19.157660 18328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:19.163548 18328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:40509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:19.164695 18328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:19.165782 18328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:19.174597 18333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:19.174600 18336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:19.174597 18334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:19.174846 18328 server_base.cc:1034] running on GCE node
I20260923 18:00:19.175661 18328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:19.176774 18328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:19.177963 18328 hybrid_clock.cc:648] HybridClock initialized: now 1790186419177949 us; error 28 us; skew 500 ppm
Sep 23 18:00:19 dist-test-slave-vr57 krb5kdc[18315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186419, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:19.180792 18328 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:00:19.181669 18328 webserver.cc:460] Webserver started at http://127.15.139.190:38037/ using document root <none> and password file <none>
I20260923 18:00:19.182317 18328 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:19.182410 18328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:19.182648 18328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:19.184816 18328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "e31dc96a0208485a94155db20e81d767"
format_stamp: "Formatted at 2026-09-23 18:00:19 on dist-test-slave-vr57"
server_key: "3537a1282518b89c61f8a5bcdf34c8d6"
server_key_iv: "50adcaf99617348b2ae1b056d2206d3f"
server_key_version: "encryptionkey@0"
I20260923 18:00:19.185405 18328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "e31dc96a0208485a94155db20e81d767"
format_stamp: "Formatted at 2026-09-23 18:00:19 on dist-test-slave-vr57"
server_key: "3537a1282518b89c61f8a5bcdf34c8d6"
server_key_iv: "50adcaf99617348b2ae1b056d2206d3f"
server_key_version: "encryptionkey@0"
I20260923 18:00:19.189158 18328 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260923 18:00:19.191789 18343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:19.193132 18328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:19.193305 18328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "e31dc96a0208485a94155db20e81d767"
format_stamp: "Formatted at 2026-09-23 18:00:19 on dist-test-slave-vr57"
server_key: "3537a1282518b89c61f8a5bcdf34c8d6"
server_key_iv: "50adcaf99617348b2ae1b056d2206d3f"
server_key_version: "encryptionkey@0"
I20260923 18:00:19.193425 18328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:19.208261 18328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:19.212185 18328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:19.212409 18328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:19.219990 18328 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:40509
I20260923 18:00:19.220050 18395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:40509 every 8 connection(s)
I20260923 18:00:19.221024 18328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:19.224047 18396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:19.225546 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 18328
I20260923 18:00:19.225654 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:19.225960 15918 external_mini_cluster.cc:1428] Setting key 1f1d8b020f3292b64bd28f96f51ee2fc
I20260923 18:00:19.230170 18396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767: Bootstrap starting.
I20260923 18:00:19.232522 18396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:19.233249 18396 log.cc:824] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767: Log is configured to *not* fsync() on all Append() calls
Sep 23 18:00:19 dist-test-slave-vr57 krb5kdc[18315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:19.235566 18396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767: No bootstrap required, opened a new log
I20260923 18:00:19.239889 18396 raft_consensus.cc:348] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31dc96a0208485a94155db20e81d767" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40509 } }
I20260923 18:00:19.240135 18396 raft_consensus.cc:374] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:19.240193 18396 raft_consensus.cc:729] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e31dc96a0208485a94155db20e81d767, State: Initialized, Role: FOLLOWER
I20260923 18:00:19.240556 18399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:19.227439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42648 (local address 127.15.139.190:40509)
0923 18:00:19.227904 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:19.227920 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:19.227964 (+ 44us) server_negotiation.cc:407] Connection header received
0923 18:00:19.228612 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:19.228628 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:19.228954 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:19.229281 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:19.230103 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.231106 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.232067 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.232552 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.232724 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:19.235574 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:19.235579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:19.235592 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:19.235639 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:19.237325 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:19.237905 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:19.237910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:19.237914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:19.238036 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:19.238338 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:19.238340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:19.238341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:19.238732 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:19.238887 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:19.239205 (+ 318us) server_negotiation.cc:299] Negotiation successful
0923 18:00:19.239608 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":140,"threads_started":1}
I20260923 18:00:19.240681 18396 consensus_queue.cc:260] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31dc96a0208485a94155db20e81d767" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40509 } }
I20260923 18:00:19.240875 18396 raft_consensus.cc:388] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:19.240928 18396 raft_consensus.cc:482] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:19.241015 18396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:19.242126 18396 raft_consensus.cc:504] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31dc96a0208485a94155db20e81d767" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40509 } }
I20260923 18:00:19.242507 18396 leader_election.cc:302] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e31dc96a0208485a94155db20e81d767; no voters:
I20260923 18:00:19.242910 18396 leader_election.cc:288] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:19.242988 18401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:19.243173 18401 raft_consensus.cc:686] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [term 1 LEADER]: Becoming Leader. State: Replica: e31dc96a0208485a94155db20e81d767, State: Running, Role: LEADER
I20260923 18:00:19.243474 18401 consensus_queue.cc:237] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31dc96a0208485a94155db20e81d767" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40509 } }
I20260923 18:00:19.244251 18396 sys_catalog.cc:564] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:19.245241 18403 sys_catalog.cc:455] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e31dc96a0208485a94155db20e81d767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31dc96a0208485a94155db20e81d767" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40509 } } }
I20260923 18:00:19.245368 18403 sys_catalog.cc:458] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:19.245772 18410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:19.245728 18402 sys_catalog.cc:455] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e31dc96a0208485a94155db20e81d767. Latest consensus state: current_term: 1 leader_uuid: "e31dc96a0208485a94155db20e81d767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31dc96a0208485a94155db20e81d767" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40509 } } }
I20260923 18:00:19.245932 18402 sys_catalog.cc:458] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:19.249107 18410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:19.254917 18410 catalog_manager.cc:1347] Generated new cluster ID: a66e663f9be14954bec0fd045359f2c9
I20260923 18:00:19.254995 18410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:19.267959 18410 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:19.269030 18410 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:19.278345 18410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e31dc96a0208485a94155db20e81d767: Generated new TSK 0
I20260923 18:00:19.279124 18410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:00:19.312332 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40509
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:19.433909 18422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:19.434186 18422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:19.434232 18422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:19.438308 18422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:19.438378 18422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:19.438441 18422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:19.443886 18422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:19.445048 18422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:19.446204 18422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:19.456275 18430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:19.456298 18428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:19.456360 18427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:19.457863 18422 server_base.cc:1034] running on GCE node
I20260923 18:00:19.458348 18422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:19.459131 18422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:19.460297 18422 hybrid_clock.cc:648] HybridClock initialized: now 1790186419460244 us; error 67 us; skew 500 ppm
Sep 23 18:00:19 dist-test-slave-vr57 krb5kdc[18315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186419, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:19.463301 18422 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:00:19.464433 18422 webserver.cc:460] Webserver started at http://127.15.139.129:42253/ using document root <none> and password file <none>
I20260923 18:00:19.465102 18422 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:19.465173 18422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:19.465349 18422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:19.467607 18422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "abe6bd4d242646b295bab0224cd95891"
format_stamp: "Formatted at 2026-09-23 18:00:19 on dist-test-slave-vr57"
server_key: "e13da3e7d06404b4fe8275ceb6763542"
server_key_iv: "6faafda54a9243bea45d974f10e32ab5"
server_key_version: "encryptionkey@0"
I20260923 18:00:19.468228 18422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "abe6bd4d242646b295bab0224cd95891"
format_stamp: "Formatted at 2026-09-23 18:00:19 on dist-test-slave-vr57"
server_key: "e13da3e7d06404b4fe8275ceb6763542"
server_key_iv: "6faafda54a9243bea45d974f10e32ab5"
server_key_version: "encryptionkey@0"
I20260923 18:00:19.472282 18422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260923 18:00:19.474965 18437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:19.476534 18422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260923 18:00:19.476650 18422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "abe6bd4d242646b295bab0224cd95891"
format_stamp: "Formatted at 2026-09-23 18:00:19 on dist-test-slave-vr57"
server_key: "e13da3e7d06404b4fe8275ceb6763542"
server_key_iv: "6faafda54a9243bea45d974f10e32ab5"
server_key_version: "encryptionkey@0"
I20260923 18:00:19.476742 18422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:19.493211 18422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:19.496743 18422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:19.496949 18422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:19.497545 18422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:19.498391 18422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:19.498447 18422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:19.498488 18422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:19.498509 18422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:19.507582 18422 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:36937
I20260923 18:00:19.507611 18550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:36937 every 8 connection(s)
I20260923 18:00:19.508497 18422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:19.509960 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 18422
I20260923 18:00:19.510066 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:19.510389 15918 external_mini_cluster.cc:1428] Setting key cb1789cdfa4e2e9ed4a85fe49c5c1f68
Sep 23 18:00:19 dist-test-slave-vr57 krb5kdc[18315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186419, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:00:19.524693 18399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:19.510473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:53473 (local address 127.15.139.190:40509)
0923 18:00:19.510612 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:19.510617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:19.511829 (+ 1212us) server_negotiation.cc:407] Connection header received
0923 18:00:19.513705 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:19.513708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:19.513787 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:19.513903 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:19.515124 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.516070 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.517330 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.517806 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.517968 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:19.520944 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:19.520947 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:19.520950 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:19.520973 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:19.522438 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:19.522910 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:19.522912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:19.522914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:19.522986 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:19.523382 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:19.523384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:19.523385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:19.523638 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:19.523755 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:19.524381 (+ 626us) server_negotiation.cc:299] Negotiation successful
0923 18:00:19.524505 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 18:00:19.525686 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:19.510967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40509 (local address 127.15.139.129:53473)
0923 18:00:19.511657 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:19.511703 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:19.513437 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:19.514110 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:19.514117 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:19.514469 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:19.514987 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:19.515000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.516204 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.516207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:19.517183 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:19.517193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.517980 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.517983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:19.518139 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:19.518807 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:19.518823 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:19.520720 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:19.522549 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:19.522553 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:19.522566 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:19.522820 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:19.523117 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:19.523120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:19.523123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:19.523299 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:19.523785 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:19.523792 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:19.524124 (+ 332us) client_negotiation.cc:769] Sending connection context
0923 18:00:19.524341 (+ 217us) client_negotiation.cc:240] Negotiation successful
0923 18:00:19.524673 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":163,"threads_started":1}
I20260923 18:00:19.527176 18551 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40509
I20260923 18:00:19.527433 18551 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:19.528095 18551 heartbeater.cc:511] Master 127.15.139.190:40509 requested a full tablet report, sending...
I20260923 18:00:19.529783 18360 ts_manager.cc:194] Registered new tserver with Master: abe6bd4d242646b295bab0224cd95891 (127.15.139.129:36937)
I20260923 18:00:19.530928 18360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:53473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:00:19.540588 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40509
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:19.653255 18556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:19.653568 18556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:19.653647 18556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:19.657835 18556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:19.657945 18556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:19.658043 18556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:19.663594 18556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:19.664737 18556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:19.665879 18556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:19.675673 18562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:19.675710 18561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:19.675887 18556 server_base.cc:1034] running on GCE node
W20260923 18:00:19.675704 18564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:19.676357 18556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:19.677033 18556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:19.678251 18556 hybrid_clock.cc:648] HybridClock initialized: now 1790186419678214 us; error 48 us; skew 500 ppm
Sep 23 18:00:19 dist-test-slave-vr57 krb5kdc[18315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186419, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:19.681385 18556 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:00:19.682358 18556 webserver.cc:460] Webserver started at http://127.15.139.130:37867/ using document root <none> and password file <none>
I20260923 18:00:19.682933 18556 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:19.683025 18556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:19.683265 18556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:19.685357 18556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "76d93121664e40a4b65559d04953170a"
format_stamp: "Formatted at 2026-09-23 18:00:19 on dist-test-slave-vr57"
server_key: "b08886be36e4701cefa77239c4bea7aa"
server_key_iv: "77cbbb9e4848489e82bc9b4460b938d9"
server_key_version: "encryptionkey@0"
I20260923 18:00:19.685940 18556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "76d93121664e40a4b65559d04953170a"
format_stamp: "Formatted at 2026-09-23 18:00:19 on dist-test-slave-vr57"
server_key: "b08886be36e4701cefa77239c4bea7aa"
server_key_iv: "77cbbb9e4848489e82bc9b4460b938d9"
server_key_version: "encryptionkey@0"
I20260923 18:00:19.689851 18556 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:19.692495 18571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:19.693795 18556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:00:19.693951 18556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "76d93121664e40a4b65559d04953170a"
format_stamp: "Formatted at 2026-09-23 18:00:19 on dist-test-slave-vr57"
server_key: "b08886be36e4701cefa77239c4bea7aa"
server_key_iv: "77cbbb9e4848489e82bc9b4460b938d9"
server_key_version: "encryptionkey@0"
I20260923 18:00:19.694068 18556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:19.717017 18556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:19.720438 18556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:19.720659 18556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:19.721330 18556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:19.722270 18556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:19.722339 18556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:19.722411 18556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:19.722460 18556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:19.732178 18556 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:36545
I20260923 18:00:19.732195 18684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:36545 every 8 connection(s)
I20260923 18:00:19.733095 18556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:19.737354 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 18556
I20260923 18:00:19.737496 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:19.737803 15918 external_mini_cluster.cc:1428] Setting key 9aa2ac941cce5a36c58d5813ee948d80
Sep 23 18:00:19 dist-test-slave-vr57 krb5kdc[18315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186419, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
I20260923 18:00:19.748843 18399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:19.734955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:48809 (local address 127.15.139.190:40509)
0923 18:00:19.735124 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:19.735128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:19.736274 (+ 1146us) server_negotiation.cc:407] Connection header received
0923 18:00:19.737283 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:19.737288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:19.737357 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:19.737494 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:19.738983 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.739751 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.740956 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.741353 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.741487 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:19.744567 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:19.744571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:19.744573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:19.744602 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:19.746488 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:19.747103 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:19.747107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:19.747109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:19.747198 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:19.747651 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:19.747656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:19.747658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:19.747933 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:19.748046 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:19.748594 (+ 548us) server_negotiation.cc:299] Negotiation successful
0923 18:00:19.748708 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260923 18:00:19.749716 18687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:19.735297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40509 (local address 127.15.139.130:48809)
0923 18:00:19.736109 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:19.736158 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:19.737025 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:19.737744 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:19.737752 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:19.738205 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:19.738828 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:19.738843 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.739891 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.739894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:19.740810 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:19.740819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.741539 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.741544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:19.741744 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:19.742418 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:19.742436 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:19.744378 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:19.746656 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:19.746662 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:19.746680 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:19.746981 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:19.747298 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:19.747302 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:19.747305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:19.747540 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:19.748055 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:19.748061 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:19.748358 (+ 297us) client_negotiation.cc:769] Sending connection context
0923 18:00:19.748523 (+ 165us) client_negotiation.cc:240] Negotiation successful
0923 18:00:19.748805 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":165,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
I20260923 18:00:19.750970 18685 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40509
I20260923 18:00:19.751235 18685 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:19.751797 18685 heartbeater.cc:511] Master 127.15.139.190:40509 requested a full tablet report, sending...
I20260923 18:00:19.752827 18360 ts_manager.cc:194] Registered new tserver with Master: 76d93121664e40a4b65559d04953170a (127.15.139.130:36545)
I20260923 18:00:19.753468 18360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:48809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:00:19.767057 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40509
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:19.879545 18690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:19.879853 18690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:19.879951 18690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:19.884244 18690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:19.884326 18690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:19.884415 18690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:19.890321 18690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:19.891487 18690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:19.892853 18690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:19.903649 18696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:19.903671 18695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:19.903791 18690 server_base.cc:1034] running on GCE node
W20260923 18:00:19.904023 18698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:19.904417 18690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:19.905066 18690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:19.906260 18690 hybrid_clock.cc:648] HybridClock initialized: now 1790186419906242 us; error 29 us; skew 500 ppm
Sep 23 18:00:19 dist-test-slave-vr57 krb5kdc[18315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186419, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:19.909089 18690 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:00:19.909983 18690 webserver.cc:460] Webserver started at http://127.15.139.131:40617/ using document root <none> and password file <none>
I20260923 18:00:19.910527 18690 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:19.910591 18690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:19.910753 18690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:19.912875 18690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "9e8010c916514f93b007fa8ef07d80a2"
format_stamp: "Formatted at 2026-09-23 18:00:19 on dist-test-slave-vr57"
server_key: "f7df9447254ab14ec47bc06e604aabf5"
server_key_iv: "8da6a38b3022b5ab721c0c38f9cf603a"
server_key_version: "encryptionkey@0"
I20260923 18:00:19.913434 18690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "9e8010c916514f93b007fa8ef07d80a2"
format_stamp: "Formatted at 2026-09-23 18:00:19 on dist-test-slave-vr57"
server_key: "f7df9447254ab14ec47bc06e604aabf5"
server_key_iv: "8da6a38b3022b5ab721c0c38f9cf603a"
server_key_version: "encryptionkey@0"
I20260923 18:00:19.917217 18690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260923 18:00:19.919852 18705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:19.921162 18690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:00:19.921268 18690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "9e8010c916514f93b007fa8ef07d80a2"
format_stamp: "Formatted at 2026-09-23 18:00:19 on dist-test-slave-vr57"
server_key: "f7df9447254ab14ec47bc06e604aabf5"
server_key_iv: "8da6a38b3022b5ab721c0c38f9cf603a"
server_key_version: "encryptionkey@0"
I20260923 18:00:19.921420 18690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:19.941144 18690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:19.944720 18690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:19.945019 18690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:19.945688 18690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:19.946628 18690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:19.946707 18690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:19.946782 18690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:19.946835 18690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:19.957041 18690 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:34745
I20260923 18:00:19.957062 18818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:34745 every 8 connection(s)
I20260923 18:00:19.958088 18690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:19.964011 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 18690
I20260923 18:00:19.964143 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:00:19.964468 15918 external_mini_cluster.cc:1428] Setting key ddf5be6d0f609b64ee51ea444a6081df
Sep 23 18:00:19 dist-test-slave-vr57 krb5kdc[18315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186419, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:19.972555 18399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:19.960160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:45537 (local address 127.15.139.190:40509)
0923 18:00:19.960304 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:19.960307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:19.961238 (+ 931us) server_negotiation.cc:407] Connection header received
0923 18:00:19.962547 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:19.962551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:19.962611 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:19.962690 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:19.963925 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.964661 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.966164 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.966380 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.966470 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:19.969179 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:19.969182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:19.969184 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:19.969208 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:19.970672 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:19.971129 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:19.971132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:19.971133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:19.971202 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:19.971494 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:19.971496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:19.971497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:19.971763 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:19.971840 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:19.972332 (+ 492us) server_negotiation.cc:299] Negotiation successful
0923 18:00:19.972439 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:00:19.973357 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:19.960521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40509 (local address 127.15.139.131:45537)
0923 18:00:19.961066 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:19.961110 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:19.962331 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:19.962857 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:19.962864 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:19.963226 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:19.963752 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:19.963774 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.964796 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.964804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:19.966024 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:19.966035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:19.966535 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:19.966539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:19.966786 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:19.967465 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:19.967482 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:19.968996 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:19.970775 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:19.970779 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:19.970792 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:19.971043 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:19.971282 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:19.971284 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:19.971285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:19.971417 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:19.971853 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:19.971857 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:19.972117 (+ 260us) client_negotiation.cc:769] Sending connection context
0923 18:00:19.972275 (+ 158us) client_negotiation.cc:240] Negotiation successful
0923 18:00:19.972527 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":120,"threads_started":1}
I20260923 18:00:19.974594 18819 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40509
I20260923 18:00:19.974964 18819 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:19.975585 18819 heartbeater.cc:511] Master 127.15.139.190:40509 requested a full tablet report, sending...
I20260923 18:00:19.976771 18360 ts_manager.cc:194] Registered new tserver with Master: 9e8010c916514f93b007fa8ef07d80a2 (127.15.139.131:34745)
I20260923 18:00:19.977458 18360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:45537
I20260923 18:00:19.978816 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 18:00:19.992486 18399 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:19.991131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42660 (local address 127.15.139.190:40509)
0923 18:00:19.991261 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:19.991265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:19.991322 (+ 57us) server_negotiation.cc:407] Connection header received
0923 18:00:19.991459 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:19.991462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:19.991561 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:19.991641 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:19.992369 (+ 728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42660: BlockingRecv error: recv got EOF from 127.0.0.1:42660 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:00:19.993511 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 18422
I20260923 18:00:19.999240 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 18556
I20260923 18:00:20.004897 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 18690
I20260923 18:00:20.010262 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 18328
2026-09-23T18:00:20Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2151 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:20 dist-test-slave-vr57 krb5kdc[18831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:20 dist-test-slave-vr57 krb5kdc[18831](info): set up 2 sockets
Sep 23 18:00:20 dist-test-slave-vr57 krb5kdc[18831](info): commencing operation
krb5kdc: starting...
W20260923 18:00:21.087989 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:21 dist-test-slave-vr57 krb5kdc[18831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:00:21.215400 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:37885
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:37885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:21.327237 18844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:21.327605 18844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:21.327718 18844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:21.331903 18844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:21.332001 18844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:21.332046 18844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:21.332108 18844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:21.332150 18844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:21.337424 18844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:37885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:37885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:21.338555 18844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:21.339656 18844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:21.348007 18850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:21.348083 18849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:21.348017 18852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:21.348363 18844 server_base.cc:1034] running on GCE node
I20260923 18:00:21.348891 18844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:21.350026 18844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:21.351209 18844 hybrid_clock.cc:648] HybridClock initialized: now 1790186421351194 us; error 32 us; skew 500 ppm
Sep 23 18:00:21 dist-test-slave-vr57 krb5kdc[18831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:21.358722 18844 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:00:21.359764 18844 webserver.cc:460] Webserver started at http://127.15.139.190:40001/ using document root <none> and password file <none>
I20260923 18:00:21.360338 18844 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:21.360428 18844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:21.360653 18844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:21.362715 18844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "6c15d9507e0d46eab81eda7c093d79ae"
format_stamp: "Formatted at 2026-09-23 18:00:21 on dist-test-slave-vr57"
server_key: "e873aa8792999755ec11e6d833b63dd0"
server_key_iv: "9c1fd10cdafb2dab31ebed5a4feb26f0"
server_key_version: "encryptionkey@0"
I20260923 18:00:21.363265 18844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "6c15d9507e0d46eab81eda7c093d79ae"
format_stamp: "Formatted at 2026-09-23 18:00:21 on dist-test-slave-vr57"
server_key: "e873aa8792999755ec11e6d833b63dd0"
server_key_iv: "9c1fd10cdafb2dab31ebed5a4feb26f0"
server_key_version: "encryptionkey@0"
I20260923 18:00:21.366878 18844 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:21.369376 18859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:21.370625 18844 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260923 18:00:21.370759 18844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "6c15d9507e0d46eab81eda7c093d79ae"
format_stamp: "Formatted at 2026-09-23 18:00:21 on dist-test-slave-vr57"
server_key: "e873aa8792999755ec11e6d833b63dd0"
server_key_iv: "9c1fd10cdafb2dab31ebed5a4feb26f0"
server_key_version: "encryptionkey@0"
I20260923 18:00:21.370868 18844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:21.391227 18844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:21.394151 18844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:21.394346 18844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:21.401255 18844 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:37885
I20260923 18:00:21.401537 18911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:37885 every 8 connection(s)
I20260923 18:00:21.402211 18844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:21.405012 18912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:21.410342 18912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae: Bootstrap starting.
I20260923 18:00:21.412026 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 18844
I20260923 18:00:21.412146 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:21.412412 15918 external_mini_cluster.cc:1428] Setting key c25980adb8b3bd7fc63bccf2199c17fa
I20260923 18:00:21.413087 18912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:21.413980 18912 log.cc:824] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:21.416074 18912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae: No bootstrap required, opened a new log
I20260923 18:00:21.420324 18912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c15d9507e0d46eab81eda7c093d79ae" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37885 } }
I20260923 18:00:21.420643 18912 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:21.420692 18912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c15d9507e0d46eab81eda7c093d79ae, State: Initialized, Role: FOLLOWER
I20260923 18:00:21.421175 18912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c15d9507e0d46eab81eda7c093d79ae" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37885 } }
I20260923 18:00:21.421300 18912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:21.421387 18912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:21.421482 18912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [term 0 FOLLOWER]: Advancing to term 1
Sep 23 18:00:21 dist-test-slave-vr57 krb5kdc[18831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:21.422569 18912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c15d9507e0d46eab81eda7c093d79ae" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37885 } }
I20260923 18:00:21.422974 18912 leader_election.cc:302] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c15d9507e0d46eab81eda7c093d79ae; no voters:
I20260923 18:00:21.423398 18912 leader_election.cc:288] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:21.423506 18917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:21.423774 18917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [term 1 LEADER]: Becoming Leader. State: Replica: 6c15d9507e0d46eab81eda7c093d79ae, State: Running, Role: LEADER
I20260923 18:00:21.424129 18917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c15d9507e0d46eab81eda7c093d79ae" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37885 } }
I20260923 18:00:21.424743 18912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:21.425875 18918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c15d9507e0d46eab81eda7c093d79ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c15d9507e0d46eab81eda7c093d79ae" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37885 } } }
I20260923 18:00:21.425988 18918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:21.426375 18919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c15d9507e0d46eab81eda7c093d79ae. Latest consensus state: current_term: 1 leader_uuid: "6c15d9507e0d46eab81eda7c093d79ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c15d9507e0d46eab81eda7c093d79ae" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37885 } } }
I20260923 18:00:21.426530 18919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:21.428246 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:21.413441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.15.139.190:37885)
0923 18:00:21.413847 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:21.413863 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:21.413898 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:00:21.414758 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:21.414777 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:21.415100 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:21.415376 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:21.416297 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:21.417299 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:21.418525 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:21.418977 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:21.419101 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:21.422121 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:21.422127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:21.422144 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:21.422175 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:21.424273 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:21.424783 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:21.424789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:21.424796 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:21.424939 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:21.425278 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:21.425281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:21.425283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:21.425838 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:21.426814 (+ 976us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:21.426976 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 18:00:21.427460 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":131,"threads_started":1}
W20260923 18:00:21.429970 18932 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:00:21.430047 18932 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:00:21.430176 18926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:21.431010 18926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:21.431614 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:00:21.436589 18926 catalog_manager.cc:1347] Generated new cluster ID: 44cbe578d182467e998a6ebe4e37be89
I20260923 18:00:21.436652 18926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:21.451532 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:21.445874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53376 (local address 127.15.139.190:37885)
0923 18:00:21.446005 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:21.446011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:21.446275 (+ 264us) server_negotiation.cc:407] Connection header received
0923 18:00:21.446426 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:21.446429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:21.446479 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:21.446552 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:21.447144 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:21.447927 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:21.449103 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:21.449438 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:21.449525 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:21.450449 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:21.450453 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:21.450456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:21.451049 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:21.451083 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:21.451102 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:21.451183 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:21.451299 (+ 116us) server_negotiation.cc:299] Negotiation successful
0923 18:00:21.451358 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 18:00:21.455565 18926 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:21.456709 18926 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:21.468123 18926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c15d9507e0d46eab81eda7c093d79ae: Generated new TSK 0
I20260923 18:00:21.468806 18926 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:00:21.474705 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 18844
2026-09-23T18:00:21Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1461 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:21 dist-test-slave-vr57 krb5kdc[18945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:21 dist-test-slave-vr57 krb5kdc[18945](info): set up 2 sockets
Sep 23 18:00:21 dist-test-slave-vr57 krb5kdc[18945](info): commencing operation
krb5kdc: starting...
W20260923 18:00:22.508150 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:22 dist-test-slave-vr57 krb5kdc[18945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:00:22.634162 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:37063
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:37063
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:22.746785 18958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:22.747100 18958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:22.747143 18958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:22.751323 18958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:22.751402 18958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:22.751422 18958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:22.751439 18958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:22.751456 18958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:22.757264 18958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:37063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:37063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:22.758446 18958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:22.759558 18958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:22.768333 18963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:22.768333 18966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:22.768560 18964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:22.768639 18958 server_base.cc:1034] running on GCE node
I20260923 18:00:22.769237 18958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:22.770383 18958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:22.771589 18958 hybrid_clock.cc:648] HybridClock initialized: now 1790186422771567 us; error 31 us; skew 500 ppm
Sep 23 18:00:22 dist-test-slave-vr57 krb5kdc[18945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186422, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:22.774490 18958 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:00:22.775460 18958 webserver.cc:460] Webserver started at http://127.15.139.190:41357/ using document root <none> and password file <none>
I20260923 18:00:22.776083 18958 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:22.776145 18958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:22.776368 18958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:22.778468 18958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "36f5f1a9267f4a2ba52f62c1d56490c6"
format_stamp: "Formatted at 2026-09-23 18:00:22 on dist-test-slave-vr57"
server_key: "09ccb43e1809d283319ed6e5f63f3dcf"
server_key_iv: "748f7de9013efec323d9a812ad46becd"
server_key_version: "encryptionkey@0"
I20260923 18:00:22.779081 18958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "36f5f1a9267f4a2ba52f62c1d56490c6"
format_stamp: "Formatted at 2026-09-23 18:00:22 on dist-test-slave-vr57"
server_key: "09ccb43e1809d283319ed6e5f63f3dcf"
server_key_iv: "748f7de9013efec323d9a812ad46becd"
server_key_version: "encryptionkey@0"
I20260923 18:00:22.782912 18958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:00:22.785513 18973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:22.786782 18958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:22.786926 18958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "36f5f1a9267f4a2ba52f62c1d56490c6"
format_stamp: "Formatted at 2026-09-23 18:00:22 on dist-test-slave-vr57"
server_key: "09ccb43e1809d283319ed6e5f63f3dcf"
server_key_iv: "748f7de9013efec323d9a812ad46becd"
server_key_version: "encryptionkey@0"
I20260923 18:00:22.787048 18958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:22.804596 18958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:22.808185 18958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:22.808413 18958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:22.815830 18958 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:37063
I20260923 18:00:22.816205 19025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:37063 every 8 connection(s)
I20260923 18:00:22.816893 18958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:22.819804 19026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:22.821252 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 18958
I20260923 18:00:22.821388 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:22.821687 15918 external_mini_cluster.cc:1428] Setting key 23e69e143223f8a91bb4fccfdc1517e5
I20260923 18:00:22.826447 19026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6: Bootstrap starting.
I20260923 18:00:22.829172 19026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:22.829916 19026 log.cc:824] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6: Log is configured to *not* fsync() on all Append() calls
Sep 23 18:00:22 dist-test-slave-vr57 krb5kdc[18945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:22.832247 19026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6: No bootstrap required, opened a new log
I20260923 18:00:22.837066 19029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:22.823097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.15.139.190:37063)
0923 18:00:22.823564 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:22.823601 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:22.823638 (+ 37us) server_negotiation.cc:407] Connection header received
0923 18:00:22.824312 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:22.824331 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:22.824674 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:22.824952 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:22.825889 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:22.826996 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:22.828175 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:22.828657 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:22.828775 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:22.831569 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:22.831578 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:22.831595 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:22.831626 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:22.833659 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:22.834162 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:22.834168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:22.834174 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:22.834318 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:22.834652 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:22.834655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:22.834657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:22.835232 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:22.835419 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:22.835747 (+ 328us) server_negotiation.cc:299] Negotiation successful
0923 18:00:22.836053 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":137,"threads_started":1}
I20260923 18:00:22.837042 19026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f5f1a9267f4a2ba52f62c1d56490c6" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37063 } }
I20260923 18:00:22.837366 19026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:22.837424 19026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f5f1a9267f4a2ba52f62c1d56490c6, State: Initialized, Role: FOLLOWER
I20260923 18:00:22.837885 19026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f5f1a9267f4a2ba52f62c1d56490c6" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37063 } }
I20260923 18:00:22.838053 19026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:22.838120 19026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:22.838223 19026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:22.839404 19026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f5f1a9267f4a2ba52f62c1d56490c6" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37063 } }
I20260923 18:00:22.839804 19026 leader_election.cc:302] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36f5f1a9267f4a2ba52f62c1d56490c6; no voters:
I20260923 18:00:22.840137 19026 leader_election.cc:288] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:22.840281 19031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:22.840519 19031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [term 1 LEADER]: Becoming Leader. State: Replica: 36f5f1a9267f4a2ba52f62c1d56490c6, State: Running, Role: LEADER
I20260923 18:00:22.840862 19031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f5f1a9267f4a2ba52f62c1d56490c6" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37063 } }
I20260923 18:00:22.841506 19026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:22.842659 19033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36f5f1a9267f4a2ba52f62c1d56490c6. Latest consensus state: current_term: 1 leader_uuid: "36f5f1a9267f4a2ba52f62c1d56490c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f5f1a9267f4a2ba52f62c1d56490c6" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37063 } } }
I20260923 18:00:22.843011 19033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:22.842918 19032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36f5f1a9267f4a2ba52f62c1d56490c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f5f1a9267f4a2ba52f62c1d56490c6" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37063 } } }
I20260923 18:00:22.843341 19032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:22.843462 19042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:22.846350 19042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:22.852232 19042 catalog_manager.cc:1347] Generated new cluster ID: 67ba2bc14062483488edbe1c25d26b75
I20260923 18:00:22.852309 19042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:22.868587 19042 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:22.869541 19042 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:22.873062 19042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36f5f1a9267f4a2ba52f62c1d56490c6: Generated new TSK 0
I20260923 18:00:22.873883 19042 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:00:22.911224 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:37063
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:23.029090 19052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:23.029364 19052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:23.029408 19052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:23.033456 19052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:23.033531 19052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:23.033591 19052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:23.039039 19052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:37063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:23.040123 19052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:23.041095 19052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:23.051069 19057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:23.051113 19060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:23.051085 19058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:23.051358 19052 server_base.cc:1034] running on GCE node
I20260923 18:00:23.051813 19052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:23.052757 19052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:23.053937 19052 hybrid_clock.cc:648] HybridClock initialized: now 1790186423053908 us; error 42 us; skew 500 ppm
Sep 23 18:00:23 dist-test-slave-vr57 krb5kdc[18945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186423, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:23.057054 19052 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:00:23.058003 19052 webserver.cc:460] Webserver started at http://127.15.139.129:42095/ using document root <none> and password file <none>
I20260923 18:00:23.058601 19052 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:23.058667 19052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:23.058898 19052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:23.061159 19052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "5c87d49015a84a4781a777e57e7c8d36"
format_stamp: "Formatted at 2026-09-23 18:00:23 on dist-test-slave-vr57"
server_key: "91fd61ce2f5a4ff6f7f12ac46a1c69c3"
server_key_iv: "eba16b5f889053d08fbe19003e4e80f8"
server_key_version: "encryptionkey@0"
I20260923 18:00:23.061769 19052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "5c87d49015a84a4781a777e57e7c8d36"
format_stamp: "Formatted at 2026-09-23 18:00:23 on dist-test-slave-vr57"
server_key: "91fd61ce2f5a4ff6f7f12ac46a1c69c3"
server_key_iv: "eba16b5f889053d08fbe19003e4e80f8"
server_key_version: "encryptionkey@0"
I20260923 18:00:23.066076 19052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:23.068885 19067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:23.070333 19052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:00:23.070482 19052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "5c87d49015a84a4781a777e57e7c8d36"
format_stamp: "Formatted at 2026-09-23 18:00:23 on dist-test-slave-vr57"
server_key: "91fd61ce2f5a4ff6f7f12ac46a1c69c3"
server_key_iv: "eba16b5f889053d08fbe19003e4e80f8"
server_key_version: "encryptionkey@0"
I20260923 18:00:23.070631 19052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:23.086325 19052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:23.089439 19052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:23.089651 19052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:23.090319 19052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:23.091151 19052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:23.091223 19052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:23.091334 19052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:23.091380 19052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:23.101482 19052 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:38021
I20260923 18:00:23.101516 19180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:38021 every 8 connection(s)
I20260923 18:00:23.102408 19052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:23.107920 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 19052
I20260923 18:00:23.108064 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:23.108368 15918 external_mini_cluster.cc:1428] Setting key bbd74be4057065dcdddb00ee403643e9
Sep 23 18:00:23 dist-test-slave-vr57 krb5kdc[18945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186423, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:23.118402 19029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:23.104500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:57855 (local address 127.15.139.190:37063)
0923 18:00:23.104674 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:23.104679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:23.105594 (+ 915us) server_negotiation.cc:407] Connection header received
0923 18:00:23.106977 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:23.106980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:23.107029 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:23.107109 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:23.108598 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.109217 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.110578 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.110821 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.110986 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:23.114183 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:23.114190 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:23.114197 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:23.114233 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:23.116085 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:23.116645 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:23.116649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:23.116651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:23.116748 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:23.117135 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:23.117139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:23.117141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:23.117396 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:23.117509 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:23.118006 (+ 497us) server_negotiation.cc:299] Negotiation successful
0923 18:00:23.118177 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
I20260923 18:00:23.119217 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:23.104955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:37063 (local address 127.15.139.129:57855)
0923 18:00:23.105438 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:23.105483 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:23.106763 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:23.107278 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:23.107284 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:23.107786 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:23.108395 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:23.108435 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.109356 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.109359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:23.110431 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:23.110445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.111066 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.111072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:23.111318 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:23.112147 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:23.112164 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:23.113953 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:23.116208 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:23.116213 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:23.116227 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:23.116492 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:23.116861 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:23.116863 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:23.116869 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:23.117012 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:23.117504 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:23.117509 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:23.117756 (+ 247us) client_negotiation.cc:769] Sending connection context
0923 18:00:23.117916 (+ 160us) client_negotiation.cc:240] Negotiation successful
0923 18:00:23.118212 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":99,"threads_started":1}
I20260923 18:00:23.120551 19181 heartbeater.cc:347] Connected to a master server at 127.15.139.190:37063
I20260923 18:00:23.120831 19181 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:00:23.121425 19181 heartbeater.cc:511] Master 127.15.139.190:37063 requested a full tablet report, sending...
I20260923 18:00:23.123063 18990 ts_manager.cc:194] Registered new tserver with Master: 5c87d49015a84a4781a777e57e7c8d36 (127.15.139.129:38021)
I20260923 18:00:23.124584 18990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:57855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:00:23.139075 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:37063
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:23.251528 19186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:23.251819 19186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:23.251868 19186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:23.256026 19186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:23.256091 19186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:23.256199 19186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:23.262116 19186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:37063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:23.263280 19186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:23.264364 19186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:23.274086 19194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:23.274093 19191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:23.274247 19186 server_base.cc:1034] running on GCE node
W20260923 18:00:23.274159 19192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:23.274811 19186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:23.275483 19186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:23.276717 19186 hybrid_clock.cc:648] HybridClock initialized: now 1790186423276669 us; error 57 us; skew 500 ppm
Sep 23 18:00:23 dist-test-slave-vr57 krb5kdc[18945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186423, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:23.279445 19186 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:00:23.280354 19186 webserver.cc:460] Webserver started at http://127.15.139.130:43123/ using document root <none> and password file <none>
I20260923 18:00:23.280925 19186 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:23.280987 19186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:23.281224 19186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:23.283291 19186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "63caa2cc41df4a6caaa47393ee80965f"
format_stamp: "Formatted at 2026-09-23 18:00:23 on dist-test-slave-vr57"
server_key: "c73f0edda3fae43905f6aede9aa03fdc"
server_key_iv: "caa5088a48ba82106333b6ff4f5ad5e9"
server_key_version: "encryptionkey@0"
I20260923 18:00:23.283900 19186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "63caa2cc41df4a6caaa47393ee80965f"
format_stamp: "Formatted at 2026-09-23 18:00:23 on dist-test-slave-vr57"
server_key: "c73f0edda3fae43905f6aede9aa03fdc"
server_key_iv: "caa5088a48ba82106333b6ff4f5ad5e9"
server_key_version: "encryptionkey@0"
I20260923 18:00:23.287758 19186 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260923 18:00:23.290249 19201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:23.291646 19186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:23.291779 19186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "63caa2cc41df4a6caaa47393ee80965f"
format_stamp: "Formatted at 2026-09-23 18:00:23 on dist-test-slave-vr57"
server_key: "c73f0edda3fae43905f6aede9aa03fdc"
server_key_iv: "caa5088a48ba82106333b6ff4f5ad5e9"
server_key_version: "encryptionkey@0"
I20260923 18:00:23.291900 19186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:23.313776 19186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:23.317077 19186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:23.317296 19186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:23.317915 19186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:23.318760 19186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:23.318841 19186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:23.318917 19186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:23.318970 19186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:23.328126 19186 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:38659
I20260923 18:00:23.328162 19314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:38659 every 8 connection(s)
I20260923 18:00:23.329074 19186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:23.336163 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 19186
I20260923 18:00:23.336290 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:23.336583 15918 external_mini_cluster.cc:1428] Setting key ed1524f789d0ce132fdc84f4b08a15f6
Sep 23 18:00:23 dist-test-slave-vr57 krb5kdc[18945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186423, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:23.344602 19029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:23.331092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:41079 (local address 127.15.139.190:37063)
0923 18:00:23.331241 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:23.331246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:23.332225 (+ 979us) server_negotiation.cc:407] Connection header received
0923 18:00:23.333654 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:23.333657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:23.333703 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:23.333779 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:23.334963 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.335484 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.337039 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.337350 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.337465 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:23.340411 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:23.340419 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:23.340421 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:23.340444 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:23.342005 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:23.342603 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:23.342608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:23.342610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:23.342690 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:23.343057 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:23.343062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:23.343064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:23.343264 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:23.343371 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:23.344324 (+ 953us) server_negotiation.cc:299] Negotiation successful
0923 18:00:23.344435 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 18:00:23.345218 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:23.331622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:37063 (local address 127.15.139.130:41079)
0923 18:00:23.332068 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:23.332112 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:23.333438 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:23.333948 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:23.333956 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:23.334320 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:23.334837 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:23.334850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.335603 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.335605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:23.336900 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:23.336912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.337525 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.337530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:23.337744 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:23.338380 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:23.338405 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:23.340245 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:23.342121 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:23.342127 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:23.342142 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:23.342498 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:23.342824 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:23.342827 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:23.342828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:23.342959 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:23.343381 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:23.343389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:23.343692 (+ 303us) client_negotiation.cc:769] Sending connection context
0923 18:00:23.343918 (+ 226us) client_negotiation.cc:240] Negotiation successful
0923 18:00:23.344277 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
I20260923 18:00:23.346490 19315 heartbeater.cc:347] Connected to a master server at 127.15.139.190:37063
I20260923 18:00:23.346756 19315 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:23.347311 19315 heartbeater.cc:511] Master 127.15.139.190:37063 requested a full tablet report, sending...
I20260923 18:00:23.348554 18990 ts_manager.cc:194] Registered new tserver with Master: 63caa2cc41df4a6caaa47393ee80965f (127.15.139.130:38659)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
I20260923 18:00:23.349330 18990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:41079
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:00:23.364750 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:37063
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:23.480710 19320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:23.481168 19320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:23.481266 19320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:23.485510 19320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:23.485608 19320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:23.485710 19320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:23.491899 19320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:37063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:23.493063 19320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:23.494235 19320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:23.504142 19325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:23.504181 19328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:23.504145 19326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:23.504395 19320 server_base.cc:1034] running on GCE node
I20260923 18:00:23.504868 19320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:23.505621 19320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:23.506820 19320 hybrid_clock.cc:648] HybridClock initialized: now 1790186423506771 us; error 60 us; skew 500 ppm
Sep 23 18:00:23 dist-test-slave-vr57 krb5kdc[18945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186423, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:23.510041 19320 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:00:23.511106 19320 webserver.cc:460] Webserver started at http://127.15.139.131:36573/ using document root <none> and password file <none>
I20260923 18:00:23.511783 19320 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:23.511879 19320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:23.512198 19320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:23.514302 19320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "4574e244baed4a929f5a8e828d6edd9e"
format_stamp: "Formatted at 2026-09-23 18:00:23 on dist-test-slave-vr57"
server_key: "4e22e4f2ba2bbbb19d7622930c93af65"
server_key_iv: "ac5b31898ed45f4f7897e3578ff34113"
server_key_version: "encryptionkey@0"
I20260923 18:00:23.514909 19320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "4574e244baed4a929f5a8e828d6edd9e"
format_stamp: "Formatted at 2026-09-23 18:00:23 on dist-test-slave-vr57"
server_key: "4e22e4f2ba2bbbb19d7622930c93af65"
server_key_iv: "ac5b31898ed45f4f7897e3578ff34113"
server_key_version: "encryptionkey@0"
I20260923 18:00:23.518831 19320 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260923 18:00:23.521808 19335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:23.523126 19320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:23.523280 19320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "4574e244baed4a929f5a8e828d6edd9e"
format_stamp: "Formatted at 2026-09-23 18:00:23 on dist-test-slave-vr57"
server_key: "4e22e4f2ba2bbbb19d7622930c93af65"
server_key_iv: "ac5b31898ed45f4f7897e3578ff34113"
server_key_version: "encryptionkey@0"
I20260923 18:00:23.523411 19320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:23.568603 19320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:23.572494 19320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:23.572719 19320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:23.573369 19320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:23.574306 19320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:23.574388 19320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:23.574470 19320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:23.574527 19320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:23.584502 19320 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:34099
I20260923 18:00:23.584542 19448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:34099 every 8 connection(s)
I20260923 18:00:23.585472 19320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:23.592077 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 19320
I20260923 18:00:23.592203 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:00:23.592484 15918 external_mini_cluster.cc:1428] Setting key 6408ced89001919bb75c08b926b9854f
Sep 23 18:00:23 dist-test-slave-vr57 krb5kdc[18945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186423, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:23.599982 19029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:23.587531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:34769 (local address 127.15.139.190:37063)
0923 18:00:23.587662 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:23.587666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:23.588506 (+ 840us) server_negotiation.cc:407] Connection header received
0923 18:00:23.589852 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:23.589855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:23.589903 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:23.589996 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:23.591177 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.591852 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.593257 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.593498 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.593642 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:23.596479 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:23.596482 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:23.596484 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:23.596510 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:23.597992 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:23.598477 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:23.598480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:23.598481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:23.598567 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:23.598862 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:23.598864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:23.598865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:23.599103 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:23.599182 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:23.599741 (+ 559us) server_negotiation.cc:299] Negotiation successful
0923 18:00:23.599844 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 18:00:23.600834 19451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:23.587916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:37063 (local address 127.15.139.131:34769)
0923 18:00:23.588354 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:23.588395 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:23.589629 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:23.590164 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:23.590171 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:23.590566 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:23.591046 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:23.591060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.591994 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.591998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:23.593120 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:23.593130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.593679 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.593683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:23.593879 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:23.594567 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:23.594589 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:23.596291 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:23.598103 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:23.598108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:23.598122 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:23.598380 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:23.598648 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:23.598651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:23.598652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:23.598780 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:23.599191 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:23.599196 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:23.599426 (+ 230us) client_negotiation.cc:769] Sending connection context
0923 18:00:23.599663 (+ 237us) client_negotiation.cc:240] Negotiation successful
0923 18:00:23.599933 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":109,"threads_started":1}
I20260923 18:00:23.602032 19449 heartbeater.cc:347] Connected to a master server at 127.15.139.190:37063
I20260923 18:00:23.602245 19449 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:23.602726 19449 heartbeater.cc:511] Master 127.15.139.190:37063 requested a full tablet report, sending...
I20260923 18:00:23.603727 18990 ts_manager.cc:194] Registered new tserver with Master: 4574e244baed4a929f5a8e828d6edd9e (127.15.139.131:34099)
I20260923 18:00:23.604269 18990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:34769
I20260923 18:00:23.606307 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 18:00:23 dist-test-slave-vr57 krb5kdc[18945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186423, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 23 18:00:23 dist-test-slave-vr57 krb5kdc[18945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186423, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:23.632340 19029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:23.623494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.15.139.190:37063)
0923 18:00:23.623694 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:23.623699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:23.623893 (+ 194us) server_negotiation.cc:407] Connection header received
0923 18:00:23.624020 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:23.624024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:23.624074 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:23.624151 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:23.624877 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.625411 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.626265 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:23.626471 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:23.626645 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:23.629324 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:23.629327 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:23.629329 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:23.629354 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:23.630687 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:23.631104 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:23.631107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:23.631108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:23.631181 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:23.631408 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:23.631410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:23.631411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:23.631700 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:23.631807 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:23.632041 (+ 234us) server_negotiation.cc:299] Negotiation successful
0923 18:00:23.632132 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260923 18:00:23.632776 18990 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41856
I20260923 18:00:23.633575 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 19052
I20260923 18:00:23.639179 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 19186
I20260923 18:00:23.644636 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 19320
I20260923 18:00:23.649857 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 18958
2026-09-23T18:00:23Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2177 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:23 dist-test-slave-vr57 krb5kdc[19461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:23 dist-test-slave-vr57 krb5kdc[19461](info): set up 2 sockets
Sep 23 18:00:23 dist-test-slave-vr57 krb5kdc[19461](info): commencing operation
krb5kdc: starting...
W20260923 18:00:24.692102 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:24 dist-test-slave-vr57 krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:00:24.814291 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:38691
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:38691
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:24.926455 19474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:24.926767 19474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:24.926810 19474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:24.930943 19474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:24.931015 19474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:24.931034 19474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:24.931058 19474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:24.931075 19474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:24.936609 19474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:38691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:38691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:24.937863 19474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:24.938920 19474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:24.947387 19482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:24.947422 19479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:24.947443 19480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:24.947834 19474 server_base.cc:1034] running on GCE node
I20260923 18:00:24.948375 19474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:24.949465 19474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:24.950627 19474 hybrid_clock.cc:648] HybridClock initialized: now 1790186424950613 us; error 26 us; skew 500 ppm
Sep 23 18:00:24 dist-test-slave-vr57 krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186424, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:24.953467 19474 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:00:24.954389 19474 webserver.cc:460] Webserver started at http://127.15.139.190:41971/ using document root <none> and password file <none>
I20260923 18:00:24.954923 19474 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:24.954981 19474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:24.955150 19474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:24.957223 19474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "367884ebecc34c72b8005bebc4083274"
format_stamp: "Formatted at 2026-09-23 18:00:24 on dist-test-slave-vr57"
server_key: "28c083c5a49863bd74e659d4a4f65365"
server_key_iv: "118f877a786fc30f4b75e945ba9a7748"
server_key_version: "encryptionkey@0"
I20260923 18:00:24.957805 19474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "367884ebecc34c72b8005bebc4083274"
format_stamp: "Formatted at 2026-09-23 18:00:24 on dist-test-slave-vr57"
server_key: "28c083c5a49863bd74e659d4a4f65365"
server_key_iv: "118f877a786fc30f4b75e945ba9a7748"
server_key_version: "encryptionkey@0"
I20260923 18:00:24.961722 19474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260923 18:00:24.964257 19489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:24.965552 19474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:24.965695 19474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "367884ebecc34c72b8005bebc4083274"
format_stamp: "Formatted at 2026-09-23 18:00:24 on dist-test-slave-vr57"
server_key: "28c083c5a49863bd74e659d4a4f65365"
server_key_iv: "118f877a786fc30f4b75e945ba9a7748"
server_key_version: "encryptionkey@0"
I20260923 18:00:24.965811 19474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:24.983810 19474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:24.988162 19474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:24.988381 19474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:24.995407 19474 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:38691
I20260923 18:00:24.995783 19541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:38691 every 8 connection(s)
I20260923 18:00:24.996454 19474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:24.999631 19542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:25.001161 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 19474
I20260923 18:00:25.001305 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:25.001587 15918 external_mini_cluster.cc:1428] Setting key 02eaa9ef8eb249975ecc73fe8edc794f
I20260923 18:00:25.006999 19542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274: Bootstrap starting.
I20260923 18:00:25.009974 19542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:25.010809 19542 log.cc:824] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274: Log is configured to *not* fsync() on all Append() calls
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:25.013466 19542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274: No bootstrap required, opened a new log
I20260923 18:00:25.017650 19542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367884ebecc34c72b8005bebc4083274" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 38691 } }
I20260923 18:00:25.017876 19542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:25.017946 19542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 367884ebecc34c72b8005bebc4083274, State: Initialized, Role: FOLLOWER
I20260923 18:00:25.018154 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:25.003050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47314 (local address 127.15.139.190:38691)
0923 18:00:25.003528 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:25.003550 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:25.003596 (+ 46us) server_negotiation.cc:407] Connection header received
0923 18:00:25.004499 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:25.004523 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:25.004979 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:25.005370 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:25.006075 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.007770 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.008671 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.009437 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.009612 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:25.012890 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:25.012897 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:25.012989 (+ 92us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:25.013063 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:25.015030 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:25.015645 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:25.015650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:25.015654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:25.015769 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:25.016087 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:25.016089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:25.016091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:25.016483 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:25.016644 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:25.016946 (+ 302us) server_negotiation.cc:299] Negotiation successful
0923 18:00:25.017279 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":114,"threads_started":1}
I20260923 18:00:25.018464 19542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367884ebecc34c72b8005bebc4083274" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 38691 } }
I20260923 18:00:25.018640 19542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:25.018728 19542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:25.018829 19542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:25.020013 19542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367884ebecc34c72b8005bebc4083274" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 38691 } }
I20260923 18:00:25.020370 19542 leader_election.cc:302] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 367884ebecc34c72b8005bebc4083274; no voters:
I20260923 18:00:25.020752 19542 leader_election.cc:288] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:25.020851 19547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:25.021090 19547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [term 1 LEADER]: Becoming Leader. State: Replica: 367884ebecc34c72b8005bebc4083274, State: Running, Role: LEADER
I20260923 18:00:25.021423 19547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367884ebecc34c72b8005bebc4083274" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 38691 } }
I20260923 18:00:25.021981 19542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:25.023053 19548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "367884ebecc34c72b8005bebc4083274" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367884ebecc34c72b8005bebc4083274" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 38691 } } }
I20260923 18:00:25.023177 19548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:25.023452 19549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 367884ebecc34c72b8005bebc4083274. Latest consensus state: current_term: 1 leader_uuid: "367884ebecc34c72b8005bebc4083274" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367884ebecc34c72b8005bebc4083274" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 38691 } } }
I20260923 18:00:25.023561 19549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:25.023582 19556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:25.026625 19556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:25.032565 19556 catalog_manager.cc:1347] Generated new cluster ID: 664c9e698b16424688cc687dfb6085a3
I20260923 18:00:25.032634 19556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:25.041617 19556 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:25.042596 19556 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:25.058246 19556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 367884ebecc34c72b8005bebc4083274: Generated new TSK 0
I20260923 18:00:25.059031 19556 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:00:25.095798 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:38691
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:25.233035 19568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:25.233314 19568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:25.233384 19568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:25.237660 19568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:25.237730 19568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:25.237823 19568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:25.243659 19568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:38691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:25.244788 19568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:25.245842 19568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:25.255822 19573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:25.255872 19574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:25.255858 19576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:25.256057 19568 server_base.cc:1034] running on GCE node
I20260923 18:00:25.256505 19568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:25.257205 19568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:25.258420 19568 hybrid_clock.cc:648] HybridClock initialized: now 1790186425258379 us; error 52 us; skew 500 ppm
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186425, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:25.261463 19568 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:00:25.262396 19568 webserver.cc:460] Webserver started at http://127.15.139.129:39661/ using document root <none> and password file <none>
I20260923 18:00:25.262990 19568 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:25.263054 19568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:25.263222 19568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:25.265333 19568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "5715dc6649e84213bf878f639169e22d"
format_stamp: "Formatted at 2026-09-23 18:00:25 on dist-test-slave-vr57"
server_key: "4c8470d382aa4b52f18b92358157b777"
server_key_iv: "22e9a7d971bcc7b94efa9961a77ea3cb"
server_key_version: "encryptionkey@0"
I20260923 18:00:25.265882 19568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "5715dc6649e84213bf878f639169e22d"
format_stamp: "Formatted at 2026-09-23 18:00:25 on dist-test-slave-vr57"
server_key: "4c8470d382aa4b52f18b92358157b777"
server_key_iv: "22e9a7d971bcc7b94efa9961a77ea3cb"
server_key_version: "encryptionkey@0"
I20260923 18:00:25.269887 19568 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:25.272537 19583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:25.273819 19568 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:00:25.273947 19568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "5715dc6649e84213bf878f639169e22d"
format_stamp: "Formatted at 2026-09-23 18:00:25 on dist-test-slave-vr57"
server_key: "4c8470d382aa4b52f18b92358157b777"
server_key_iv: "22e9a7d971bcc7b94efa9961a77ea3cb"
server_key_version: "encryptionkey@0"
I20260923 18:00:25.274062 19568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:25.317755 19568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:25.321272 19568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:25.321569 19568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:25.322255 19568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:25.323330 19568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:25.323382 19568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:25.323479 19568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:25.323534 19568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260923 18:00:25.333189 19568 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:38629
I20260923 18:00:25.333231 19696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:38629 every 8 connection(s)
I20260923 18:00:25.334259 19568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186425, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:25.344303 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 19568
I20260923 18:00:25.344419 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:25.344743 15918 external_mini_cluster.cc:1428] Setting key 66ae5af9a8806178dba1b81fab7d9d5d
I20260923 18:00:25.348917 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:25.336107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:53185 (local address 127.15.139.190:38691)
0923 18:00:25.336257 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:25.336262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:25.337199 (+ 937us) server_negotiation.cc:407] Connection header received
0923 18:00:25.338654 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:25.338656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:25.338704 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:25.338781 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:25.340086 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.340622 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.341811 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.342031 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.342118 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:25.344805 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:25.344809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:25.344813 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:25.344844 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:25.346474 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:25.347117 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:25.347121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:25.347124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:25.347214 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:25.347669 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:25.347672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:25.347674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:25.347908 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:25.348020 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:25.348625 (+ 605us) server_negotiation.cc:299] Negotiation successful
0923 18:00:25.348755 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 18:00:25.349783 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:25.336571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:38691 (local address 127.15.139.129:53185)
0923 18:00:25.337043 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:25.337089 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:25.338461 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:25.338991 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:25.338998 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:25.339386 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:25.339946 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:25.339960 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.340762 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.340765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:25.341693 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:25.341702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.342191 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.342194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:25.342340 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:25.342966 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:25.342985 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:25.344640 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:25.346685 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:25.346691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:25.346707 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:25.346991 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:25.347315 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:25.347318 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:25.347320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:25.347483 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:25.348018 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:25.348025 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:25.348326 (+ 301us) client_negotiation.cc:769] Sending connection context
0923 18:00:25.348569 (+ 243us) client_negotiation.cc:240] Negotiation successful
0923 18:00:25.348929 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":104,"threads_started":1}
I20260923 18:00:25.351135 19697 heartbeater.cc:347] Connected to a master server at 127.15.139.190:38691
I20260923 18:00:25.351437 19697 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:25.352035 19697 heartbeater.cc:511] Master 127.15.139.190:38691 requested a full tablet report, sending...
I20260923 18:00:25.353807 19500 ts_manager.cc:194] Registered new tserver with Master: 5715dc6649e84213bf878f639169e22d (127.15.139.129:38629)
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
I20260923 18:00:25.355033 19500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:53185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:00:25.372992 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:38691
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:25.488996 19702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:25.489355 19702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:25.489444 19702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:25.493870 19702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:25.493922 19702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:25.493979 19702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:25.499261 19702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:38691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:25.500316 19702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:25.501271 19702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:25.511251 19710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:25.511343 19702 server_base.cc:1034] running on GCE node
W20260923 18:00:25.511260 19707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:25.511278 19708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:25.511991 19702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:25.512643 19702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:25.513825 19702 hybrid_clock.cc:648] HybridClock initialized: now 1790186425513785 us; error 58 us; skew 500 ppm
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186425, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:25.516718 19702 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:00:25.517625 19702 webserver.cc:460] Webserver started at http://127.15.139.130:37101/ using document root <none> and password file <none>
I20260923 18:00:25.518189 19702 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:25.518280 19702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:25.518509 19702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:25.520743 19702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "ba660fcb12c54791a2b80c49d7ffd0bb"
format_stamp: "Formatted at 2026-09-23 18:00:25 on dist-test-slave-vr57"
server_key: "a1277693935bab0d861df957206d11d2"
server_key_iv: "67bbcf32a6f1f85d3945900be5d6b9b3"
server_key_version: "encryptionkey@0"
I20260923 18:00:25.521313 19702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "ba660fcb12c54791a2b80c49d7ffd0bb"
format_stamp: "Formatted at 2026-09-23 18:00:25 on dist-test-slave-vr57"
server_key: "a1277693935bab0d861df957206d11d2"
server_key_iv: "67bbcf32a6f1f85d3945900be5d6b9b3"
server_key_version: "encryptionkey@0"
I20260923 18:00:25.525843 19702 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:25.528407 19717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:25.529728 19702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:00:25.529868 19702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "ba660fcb12c54791a2b80c49d7ffd0bb"
format_stamp: "Formatted at 2026-09-23 18:00:25 on dist-test-slave-vr57"
server_key: "a1277693935bab0d861df957206d11d2"
server_key_iv: "67bbcf32a6f1f85d3945900be5d6b9b3"
server_key_version: "encryptionkey@0"
I20260923 18:00:25.529984 19702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:25.557394 19702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:25.560962 19702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:25.561172 19702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:25.561779 19702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:25.562608 19702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:25.562676 19702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:25.562747 19702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:25.562799 19702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:25.572585 19702 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:42937
I20260923 18:00:25.572603 19830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:42937 every 8 connection(s)
I20260923 18:00:25.573494 19702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:25.579867 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 19702
I20260923 18:00:25.579998 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:25.580385 15918 external_mini_cluster.cc:1428] Setting key 8b0d5cb9b9718127ac37d37d0a473bf8
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186425, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:25.589224 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:25.575304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:60485 (local address 127.15.139.190:38691)
0923 18:00:25.575474 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:25.575478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:25.576423 (+ 945us) server_negotiation.cc:407] Connection header received
0923 18:00:25.577923 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:25.577926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:25.577973 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:25.578060 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:25.579351 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.579954 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.581566 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.581886 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.581972 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:25.585306 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:25.585310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:25.585313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:25.585344 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:25.586804 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:25.587487 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:25.587489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:25.587491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:25.587640 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:25.588075 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:25.588077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:25.588078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:25.588257 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:25.588333 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:25.589000 (+ 667us) server_negotiation.cc:299] Negotiation successful
0923 18:00:25.589103 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 18:00:25.590547 19833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:25.575777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:38691 (local address 127.15.139.130:60485)
0923 18:00:25.576255 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:25.576301 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:25.577723 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:25.578219 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:25.578225 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:25.578654 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:25.579212 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:25.579227 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.580155 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.580161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:25.581437 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:25.581447 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.582155 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.582160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:25.582363 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:25.583021 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:25.583046 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:25.585068 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:25.586929 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:25.586939 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:25.586959 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:25.587383 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:25.587744 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:25.587750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:25.587753 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:25.587973 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:25.588367 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:25.588376 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:25.588737 (+ 361us) client_negotiation.cc:769] Sending connection context
0923 18:00:25.588970 (+ 233us) client_negotiation.cc:240] Negotiation successful
0923 18:00:25.589387 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
I20260923 18:00:25.591876 19831 heartbeater.cc:347] Connected to a master server at 127.15.139.190:38691
I20260923 18:00:25.592129 19831 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:25.592603 19831 heartbeater.cc:511] Master 127.15.139.190:38691 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
I20260923 18:00:25.593837 19500 ts_manager.cc:194] Registered new tserver with Master: ba660fcb12c54791a2b80c49d7ffd0bb (127.15.139.130:42937)
I20260923 18:00:25.594414 19500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:60485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:00:25.608626 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:38691
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:25.718369 19836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:25.718617 19836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:25.718695 19836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:25.722909 19836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:25.722968 19836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:25.723063 19836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:25.728375 19836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:38691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:25.729420 19836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:25.730444 19836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:25.741930 19841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:25.741940 19844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:25.741915 19842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:25.742260 19836 server_base.cc:1034] running on GCE node
I20260923 18:00:25.742686 19836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:25.743353 19836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:25.744541 19836 hybrid_clock.cc:648] HybridClock initialized: now 1790186425744506 us; error 30 us; skew 500 ppm
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186425, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:25.747231 19836 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:00:25.748103 19836 webserver.cc:460] Webserver started at http://127.15.139.131:34901/ using document root <none> and password file <none>
I20260923 18:00:25.748629 19836 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:25.748687 19836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:25.748852 19836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:25.751085 19836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "d7563efe2c4f4eb0b018f9f29cd9924d"
format_stamp: "Formatted at 2026-09-23 18:00:25 on dist-test-slave-vr57"
server_key: "c7135c168e563341f8e7b9de22be567d"
server_key_iv: "326cde901feca6b75fee69b70db9f62c"
server_key_version: "encryptionkey@0"
I20260923 18:00:25.751729 19836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "d7563efe2c4f4eb0b018f9f29cd9924d"
format_stamp: "Formatted at 2026-09-23 18:00:25 on dist-test-slave-vr57"
server_key: "c7135c168e563341f8e7b9de22be567d"
server_key_iv: "326cde901feca6b75fee69b70db9f62c"
server_key_version: "encryptionkey@0"
I20260923 18:00:25.755455 19836 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:25.757961 19851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:25.759250 19836 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:25.759378 19836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "d7563efe2c4f4eb0b018f9f29cd9924d"
format_stamp: "Formatted at 2026-09-23 18:00:25 on dist-test-slave-vr57"
server_key: "c7135c168e563341f8e7b9de22be567d"
server_key_iv: "326cde901feca6b75fee69b70db9f62c"
server_key_version: "encryptionkey@0"
I20260923 18:00:25.759493 19836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:25.775411 19836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:25.778815 19836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:25.779043 19836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:25.779656 19836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:25.780469 19836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:25.780516 19836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:25.780558 19836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:25.780575 19836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:25.789314 19836 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:45103
I20260923 18:00:25.789374 19964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:45103 every 8 connection(s)
I20260923 18:00:25.790267 19836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:25.795284 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 19836
I20260923 18:00:25.795388 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:00:25.795742 15918 external_mini_cluster.cc:1428] Setting key ed39763ca47c196bd2cd93f408947c57
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186425, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:25.804881 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:25.792312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:47821 (local address 127.15.139.190:38691)
0923 18:00:25.792482 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:25.792487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:25.793373 (+ 886us) server_negotiation.cc:407] Connection header received
0923 18:00:25.794797 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:25.794800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:25.794850 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:25.794929 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:25.796485 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.797197 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.798464 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.798704 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.798816 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:25.801316 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:25.801320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:25.801322 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:25.801353 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:25.802683 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:25.803288 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:25.803293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:25.803295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:25.803392 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:25.803760 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:25.803764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:25.803766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:25.803955 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:25.804065 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:25.804579 (+ 514us) server_negotiation.cc:299] Negotiation successful
0923 18:00:25.804700 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260923 18:00:25.805693 19967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:25.792741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:38691 (local address 127.15.139.131:47821)
0923 18:00:25.793209 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:25.793255 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:25.794600 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:25.795132 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:25.795139 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:25.795668 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:25.796333 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:25.796350 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.797324 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.797327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:25.798304 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:25.798313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.798859 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.798863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:25.799043 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:25.799672 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:25.799688 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:25.801144 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:25.802847 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:25.802853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:25.802873 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:25.803175 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:25.803487 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:25.803492 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:25.803494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:25.803659 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:25.804073 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:25.804081 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:25.804328 (+ 247us) client_negotiation.cc:769] Sending connection context
0923 18:00:25.804522 (+ 194us) client_negotiation.cc:240] Negotiation successful
0923 18:00:25.804846 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":106,"threads_started":1}
I20260923 18:00:25.806849 19965 heartbeater.cc:347] Connected to a master server at 127.15.139.190:38691
I20260923 18:00:25.807082 19965 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:25.807649 19965 heartbeater.cc:511] Master 127.15.139.190:38691 requested a full tablet report, sending...
I20260923 18:00:25.809046 19504 ts_manager.cc:194] Registered new tserver with Master: d7563efe2c4f4eb0b018f9f29cd9924d (127.15.139.131:45103)
I20260923 18:00:25.809850 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:00:25.809839 19504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:47821
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.129@KRBTEST.COM
I20260923 18:00:25.837669 19972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:25.827598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36976 (local address 127.15.139.129:38629)
0923 18:00:25.827914 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:25.827931 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:25.827959 (+ 28us) server_negotiation.cc:407] Connection header received
0923 18:00:25.828172 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:25.828178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:25.828380 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:25.828589 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:25.829311 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.830096 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.831012 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.831401 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.831553 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:25.834074 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:25.834083 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:25.834097 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:25.834118 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:25.835891 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:25.836350 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:25.836356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:25.836357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:25.836453 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:25.836706 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:25.836708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:25.836710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:25.836979 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:25.837159 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:25.837328 (+ 169us) server_negotiation.cc:299] Negotiation successful
0923 18:00:25.837490 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":95,"threads_started":1}
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:25.849195 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:25.840882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47320 (local address 127.15.139.190:38691)
0923 18:00:25.841032 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:25.841036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:25.841141 (+ 105us) server_negotiation.cc:407] Connection header received
0923 18:00:25.841286 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:25.841290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:25.841336 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:25.841429 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:25.842027 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.842575 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.843374 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:25.843669 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:25.843778 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:25.846035 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:25.846039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:25.846041 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:25.846068 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:25.847481 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:25.847942 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:25.847945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:25.847947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:25.848033 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:25.848310 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:25.848314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:25.848315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:25.848546 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:25.848686 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:25.848916 (+ 230us) server_negotiation.cc:299] Negotiation successful
0923 18:00:25.849036 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260923 18:00:25.849753 19504 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47320
I20260923 18:00:25.850487 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 19568
I20260923 18:00:25.856163 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 19702
I20260923 18:00:25.861766 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 19836
I20260923 18:00:25.867033 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 19474
2026-09-23T18:00:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2218 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19977](info): set up 2 sockets
Sep 23 18:00:25 dist-test-slave-vr57 krb5kdc[19977](info): commencing operation
krb5kdc: starting...
W20260923 18:00:26.920131 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:27 dist-test-slave-vr57 krb5kdc[19977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:00:27.047060 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:39801
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:35243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:39801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:27.157902 19990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:27.158217 19990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:27.158303 19990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:27.162497 19990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:27.162580 19990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:27.162619 19990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:27.162678 19990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:27.162717 19990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:27.167889 19990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:35243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:39801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:39801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:27.168879 19990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:27.169872 19990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:27.178009 19996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:27.178082 19995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:27.178153 19998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:27.178048 19990 server_base.cc:1034] running on GCE node
I20260923 18:00:27.178804 19990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:27.179919 19990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:27.181114 19990 hybrid_clock.cc:648] HybridClock initialized: now 1790186427181081 us; error 46 us; skew 500 ppm
Sep 23 18:00:27 dist-test-slave-vr57 krb5kdc[19977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186427, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:27.188216 19990 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:00:27.189036 19990 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:27.189131 19990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:27.189364 19990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:27.191476 19990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "1057ce2c604b4637957ca8a99c2cca19"
format_stamp: "Formatted at 2026-09-23 18:00:27 on dist-test-slave-vr57"
server_key: "38e7155934eb01842f0872c31f5a6afe"
server_key_iv: "b03e83a26eae41f717a937183ebb29dc"
server_key_version: "encryptionkey@0"
I20260923 18:00:27.192107 19990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "1057ce2c604b4637957ca8a99c2cca19"
format_stamp: "Formatted at 2026-09-23 18:00:27 on dist-test-slave-vr57"
server_key: "38e7155934eb01842f0872c31f5a6afe"
server_key_iv: "b03e83a26eae41f717a937183ebb29dc"
server_key_version: "encryptionkey@0"
I20260923 18:00:27.195753 19990 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:27.198355 20004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:27.199786 19990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:27.199930 19990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "1057ce2c604b4637957ca8a99c2cca19"
format_stamp: "Formatted at 2026-09-23 18:00:27 on dist-test-slave-vr57"
server_key: "38e7155934eb01842f0872c31f5a6afe"
server_key_iv: "b03e83a26eae41f717a937183ebb29dc"
server_key_version: "encryptionkey@0"
I20260923 18:00:27.200049 19990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:27.216022 19990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:27.219301 19990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:27.219550 19990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:27.226596 19990 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:39801
I20260923 18:00:27.226893 20056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:39801 every 8 connection(s)
I20260923 18:00:27.227337 19990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:27.230123 20057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:27.232981 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 19990
I20260923 18:00:27.233120 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:27.233376 15918 external_mini_cluster.cc:1428] Setting key 12cd3f731ec12bae052258e9357040d4
I20260923 18:00:27.236729 20057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19: Bootstrap starting.
I20260923 18:00:27.239626 20057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:27.240451 20057 log.cc:824] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:27.242486 20057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19: No bootstrap required, opened a new log
Sep 23 18:00:27 dist-test-slave-vr57 krb5kdc[19977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:27.245687 20057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1057ce2c604b4637957ca8a99c2cca19" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 39801 } }
I20260923 18:00:27.245946 20057 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:27.246027 20057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1057ce2c604b4637957ca8a99c2cca19, State: Initialized, Role: FOLLOWER
I20260923 18:00:27.246454 20057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1057ce2c604b4637957ca8a99c2cca19" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 39801 } }
I20260923 18:00:27.246598 20057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:27.246704 20057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:27.246824 20057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:27.247802 20057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1057ce2c604b4637957ca8a99c2cca19" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 39801 } }
I20260923 18:00:27.248153 20057 leader_election.cc:302] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1057ce2c604b4637957ca8a99c2cca19; no voters:
I20260923 18:00:27.248531 20057 leader_election.cc:288] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:27.248634 20062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:27.248842 20062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [term 1 LEADER]: Becoming Leader. State: Replica: 1057ce2c604b4637957ca8a99c2cca19, State: Running, Role: LEADER
I20260923 18:00:27.249212 20062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1057ce2c604b4637957ca8a99c2cca19" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 39801 } }
I20260923 18:00:27.249609 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:27.234785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47360 (local address 127.15.139.190:39801)
0923 18:00:27.235216 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:27.235238 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:27.235285 (+ 47us) server_negotiation.cc:407] Connection header received
0923 18:00:27.236152 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:27.236175 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:27.236608 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:27.237042 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:27.237920 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.239456 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.240413 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.241058 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.241180 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:27.243757 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:27.243762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:27.243775 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:27.243802 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:27.246349 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:27.246839 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:27.246845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:27.246851 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:27.247010 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:27.247304 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:27.247308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:27.247310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:27.247783 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:27.247997 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:27.248263 (+ 266us) server_negotiation.cc:299] Negotiation successful
0923 18:00:27.248599 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":112,"threads_started":1}
I20260923 18:00:27.249799 20057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:27.250445 20062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1057ce2c604b4637957ca8a99c2cca19. Latest consensus state: current_term: 1 leader_uuid: "1057ce2c604b4637957ca8a99c2cca19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1057ce2c604b4637957ca8a99c2cca19" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 39801 } } }
I20260923 18:00:27.250649 20062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:27.250908 20063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1057ce2c604b4637957ca8a99c2cca19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1057ce2c604b4637957ca8a99c2cca19" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 39801 } } }
I20260923 18:00:27.251011 20063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:27.251067 20069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:27.254207 20069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:27.259743 20069 catalog_manager.cc:1347] Generated new cluster ID: f5c92a37674345c28460fffb9f5e351c
I20260923 18:00:27.259811 20069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:27.272022 20069 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:27.273247 20069 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:27.281428 20069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19: Generated new TSK 0
I20260923 18:00:27.282200 20069 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:00:27.319362 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:39801
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:35243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:27.434478 20083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:27.434782 20083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:27.434860 20083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:27.439028 20083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:27.439112 20083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:27.439210 20083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:27.444494 20083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:35243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:39801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:27.445587 20083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:27.446605 20083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:27.455958 20089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:27.455965 20091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:27.455992 20088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:27.456266 20083 server_base.cc:1034] running on GCE node
I20260923 18:00:27.456683 20083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:27.457244 20083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:27.458416 20083 hybrid_clock.cc:648] HybridClock initialized: now 1790186427458389 us; error 36 us; skew 500 ppm
Sep 23 18:00:27 dist-test-slave-vr57 krb5kdc[19977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186427, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:27.461292 20083 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:00:27.462061 20083 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:27.462162 20083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:27.462409 20083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:27.464520 20083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "0bdd70d5edc74610b55cba728a03d566"
format_stamp: "Formatted at 2026-09-23 18:00:27 on dist-test-slave-vr57"
server_key: "a2fce5f48971b09980890d04f3f99a74"
server_key_iv: "d8064dbbba1288da0d673386bab7f4c7"
server_key_version: "encryptionkey@0"
I20260923 18:00:27.465085 20083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "0bdd70d5edc74610b55cba728a03d566"
format_stamp: "Formatted at 2026-09-23 18:00:27 on dist-test-slave-vr57"
server_key: "a2fce5f48971b09980890d04f3f99a74"
server_key_iv: "d8064dbbba1288da0d673386bab7f4c7"
server_key_version: "encryptionkey@0"
I20260923 18:00:27.468770 20083 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:00:27.471323 20097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:27.472620 20083 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:00:27.472756 20083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "0bdd70d5edc74610b55cba728a03d566"
format_stamp: "Formatted at 2026-09-23 18:00:27 on dist-test-slave-vr57"
server_key: "a2fce5f48971b09980890d04f3f99a74"
server_key_iv: "d8064dbbba1288da0d673386bab7f4c7"
server_key_version: "encryptionkey@0"
I20260923 18:00:27.472862 20083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:27.507182 20083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:27.510349 20083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:27.510591 20083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:27.511204 20083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:27.512100 20083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:27.512146 20083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:27.512216 20083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:27.512246 20083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:27.521191 20083 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:35093
I20260923 18:00:27.521206 20210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:35093 every 8 connection(s)
I20260923 18:00:27.521797 20083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:27.526779 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 20083
I20260923 18:00:27.526919 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:27.527222 15918 external_mini_cluster.cc:1428] Setting key 88d6cfdea35b9ab3aaa3272ed9d3b05e
Sep 23 18:00:27 dist-test-slave-vr57 krb5kdc[19977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186427, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
I20260923 18:00:27.536437 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:27.523574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:35597 (local address 127.15.139.190:39801)
0923 18:00:27.523697 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:27.523700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:27.524567 (+ 867us) server_negotiation.cc:407] Connection header received
0923 18:00:27.525878 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:27.525881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:27.525938 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:27.526048 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:27.527585 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.528272 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.529930 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.530157 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.530251 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:27.532857 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:27.532860 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:27.532863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:27.532884 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:27.534345 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:27.534870 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:27.534878 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:27.534880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:27.534950 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:27.535304 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:27.535307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:27.535309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:27.535490 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:27.535646 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:27.536139 (+ 493us) server_negotiation.cc:299] Negotiation successful
0923 18:00:27.536263 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 18:00:27.537236 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:27.523966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:39801 (local address 127.15.139.129:35597)
0923 18:00:27.524392 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:27.524434 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:27.525667 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:27.526226 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:27.526232 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:27.526698 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:27.527399 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:27.527420 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.528403 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.528407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:27.529764 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:27.529778 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.530331 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.530337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:27.530537 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:27.531142 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:27.531159 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:27.532676 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:27.534476 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:27.534482 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:27.534497 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:27.534756 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:27.535049 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:27.535053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:27.535055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:27.535200 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:27.535641 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:27.535648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:27.535894 (+ 246us) client_negotiation.cc:769] Sending connection context
0923 18:00:27.536088 (+ 194us) client_negotiation.cc:240] Negotiation successful
0923 18:00:27.536411 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:00:27.538614 20211 heartbeater.cc:347] Connected to a master server at 127.15.139.190:39801
I20260923 18:00:27.538872 20211 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:27.539428 20211 heartbeater.cc:511] Master 127.15.139.190:39801 requested a full tablet report, sending...
I20260923 18:00:27.541046 20021 ts_manager.cc:194] Registered new tserver with Master: 0bdd70d5edc74610b55cba728a03d566 (127.15.139.129:35093)
I20260923 18:00:27.542284 20021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:35597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:00:27.553701 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:39801
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:35243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:27.664638 20216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:27.664904 20216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:27.664950 20216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:27.669025 20216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:27.669098 20216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:27.669159 20216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:27.674408 20216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:35243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:39801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:27.675493 20216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:27.676564 20216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:27.686026 20221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:27.686026 20224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:27.686056 20222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:27.686249 20216 server_base.cc:1034] running on GCE node
I20260923 18:00:27.686846 20216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:27.687431 20216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:27.688643 20216 hybrid_clock.cc:648] HybridClock initialized: now 1790186427688612 us; error 49 us; skew 500 ppm
Sep 23 18:00:27 dist-test-slave-vr57 krb5kdc[19977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186427, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:27.691671 20216 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:00:27.692408 20216 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:27.692503 20216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:27.692714 20216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:27.694761 20216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "4040195c15ae4da0a5d59dd6be66ab61"
format_stamp: "Formatted at 2026-09-23 18:00:27 on dist-test-slave-vr57"
server_key: "6424ad1d75562792330995d739e60e68"
server_key_iv: "2a5bf74dc022fb15bc37f3d928e830fc"
server_key_version: "encryptionkey@0"
I20260923 18:00:27.695345 20216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "4040195c15ae4da0a5d59dd6be66ab61"
format_stamp: "Formatted at 2026-09-23 18:00:27 on dist-test-slave-vr57"
server_key: "6424ad1d75562792330995d739e60e68"
server_key_iv: "2a5bf74dc022fb15bc37f3d928e830fc"
server_key_version: "encryptionkey@0"
I20260923 18:00:27.699055 20216 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:00:27.701545 20230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:27.702766 20216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:27.702883 20216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "4040195c15ae4da0a5d59dd6be66ab61"
format_stamp: "Formatted at 2026-09-23 18:00:27 on dist-test-slave-vr57"
server_key: "6424ad1d75562792330995d739e60e68"
server_key_iv: "2a5bf74dc022fb15bc37f3d928e830fc"
server_key_version: "encryptionkey@0"
I20260923 18:00:27.702994 20216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:27.726809 20216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:27.730386 20216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:27.730594 20216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:27.731209 20216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:27.732097 20216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:27.732172 20216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:27.732247 20216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:27.732290 20216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:27.742103 20216 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:43217
I20260923 18:00:27.742129 20343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:43217 every 8 connection(s)
I20260923 18:00:27.742816 20216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:27.750136 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 20216
I20260923 18:00:27.750247 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:27.750555 15918 external_mini_cluster.cc:1428] Setting key 4e0e87375f7c0db81923bffd13cc2442
Sep 23 18:00:27 dist-test-slave-vr57 krb5kdc[19977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186427, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:27.758965 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:27.744776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:57413 (local address 127.15.139.190:39801)
0923 18:00:27.744919 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:27.744923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:27.745953 (+ 1030us) server_negotiation.cc:407] Connection header received
0923 18:00:27.747308 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:27.747311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:27.747363 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:27.747468 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:27.749043 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.749703 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.751671 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.751994 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.752156 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:27.755230 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:27.755234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:27.755237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:27.755273 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:27.756867 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:27.757410 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:27.757414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:27.757416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:27.757498 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:27.757866 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:27.757869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:27.757870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:27.758060 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:27.758166 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:27.758678 (+ 512us) server_negotiation.cc:299] Negotiation successful
0923 18:00:27.758802 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 18:00:27.759966 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:27.745302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:39801 (local address 127.15.139.130:57413)
0923 18:00:27.745775 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:27.745822 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:27.747074 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:27.747750 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:27.747757 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:27.748152 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:27.748830 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:27.748853 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.749913 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.749917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:27.751483 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:27.751497 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.752229 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.752235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:27.752479 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:27.753213 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:27.753233 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:27.755025 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:27.756996 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:27.757003 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:27.757022 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:27.757302 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:27.757591 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:27.757596 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:27.757601 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:27.757761 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:27.758165 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:27.758173 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:27.758433 (+ 260us) client_negotiation.cc:769] Sending connection context
0923 18:00:27.758637 (+ 204us) client_negotiation.cc:240] Negotiation successful
0923 18:00:27.758980 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
I20260923 18:00:27.761296 20344 heartbeater.cc:347] Connected to a master server at 127.15.139.190:39801
I20260923 18:00:27.761607 20344 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:27.762173 20344 heartbeater.cc:511] Master 127.15.139.190:39801 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
I20260923 18:00:27.763590 20021 ts_manager.cc:194] Registered new tserver with Master: 4040195c15ae4da0a5d59dd6be66ab61 (127.15.139.130:43217)
I20260923 18:00:27.764472 20021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:57413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:00:27.782224 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:39801
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:35243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:27.908066 20349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:27.908372 20349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:27.908454 20349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:27.912544 20349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:27.912632 20349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:27.912719 20349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:27.918205 20349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:35243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:39801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:27.919304 20349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:27.920418 20349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:27.929916 20354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:27.929927 20357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:27.929913 20355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:27.930246 20349 server_base.cc:1034] running on GCE node
I20260923 18:00:27.930735 20349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:27.931371 20349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:27.932569 20349 hybrid_clock.cc:648] HybridClock initialized: now 1790186427932534 us; error 48 us; skew 500 ppm
Sep 23 18:00:27 dist-test-slave-vr57 krb5kdc[19977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186427, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:27.935401 20349 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:00:27.936142 20349 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:27.936247 20349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:27.936487 20349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:27.938697 20349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "782b7da69d0b43b790a187c0770aeb09"
format_stamp: "Formatted at 2026-09-23 18:00:27 on dist-test-slave-vr57"
server_key: "d7a933151fc1b3a4baf0c3fc9e8b0bbb"
server_key_iv: "b608f855bb40e745a7f2968845c9e56e"
server_key_version: "encryptionkey@0"
I20260923 18:00:27.939293 20349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "782b7da69d0b43b790a187c0770aeb09"
format_stamp: "Formatted at 2026-09-23 18:00:27 on dist-test-slave-vr57"
server_key: "d7a933151fc1b3a4baf0c3fc9e8b0bbb"
server_key_iv: "b608f855bb40e745a7f2968845c9e56e"
server_key_version: "encryptionkey@0"
I20260923 18:00:27.943106 20349 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:00:27.945793 20363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:27.947108 20349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260923 18:00:27.947256 20349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "782b7da69d0b43b790a187c0770aeb09"
format_stamp: "Formatted at 2026-09-23 18:00:27 on dist-test-slave-vr57"
server_key: "d7a933151fc1b3a4baf0c3fc9e8b0bbb"
server_key_iv: "b608f855bb40e745a7f2968845c9e56e"
server_key_version: "encryptionkey@0"
I20260923 18:00:27.947377 20349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:27.961731 20349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:27.964915 20349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:27.965126 20349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:27.965694 20349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:27.966514 20349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:27.966593 20349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:27.966696 20349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:27.966744 20349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:27.976212 20349 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:41229
I20260923 18:00:27.976272 20476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:41229 every 8 connection(s)
I20260923 18:00:27.976886 20349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:27.979142 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 20349
I20260923 18:00:27.979317 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:00:27.979769 15918 external_mini_cluster.cc:1428] Setting key fd83193f35eb998e90dae9d6b4a12191
Sep 23 18:00:27 dist-test-slave-vr57 krb5kdc[19977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186427, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:27.991734 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:27.978755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:43397 (local address 127.15.139.190:39801)
0923 18:00:27.979050 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:27.979057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:27.980161 (+ 1104us) server_negotiation.cc:407] Connection header received
0923 18:00:27.982089 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:27.982093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:27.982150 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:27.982224 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:27.983456 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.984047 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.985237 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.985487 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.985598 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:27.988209 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:27.988213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:27.988215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:27.988240 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:27.989607 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:27.990147 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:27.990151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:27.990153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:27.990236 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:27.990612 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:27.990615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:27.990616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:27.990791 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:27.990870 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:27.991394 (+ 524us) server_negotiation.cc:299] Negotiation successful
0923 18:00:27.991572 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260923 18:00:27.992587 20479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:27.979132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:39801 (local address 127.15.139.131:43397)
0923 18:00:27.979958 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:27.980025 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:27.981820 (+ 1795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:27.982403 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:27.982411 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:27.982783 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:27.983292 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:27.983306 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.984186 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.984195 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:27.985107 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:27.985116 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:27.985650 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:27.985654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:27.985827 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:27.986388 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:27.986405 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:27.988033 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:27.989714 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:27.989718 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:27.989731 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:27.990048 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:27.990328 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:27.990331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:27.990332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:27.990521 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:27.990884 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:27.990888 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:27.991179 (+ 291us) client_negotiation.cc:769] Sending connection context
0923 18:00:27.991343 (+ 164us) client_negotiation.cc:240] Negotiation successful
0923 18:00:27.991686 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":146,"threads_started":1}
I20260923 18:00:27.993846 20477 heartbeater.cc:347] Connected to a master server at 127.15.139.190:39801
I20260923 18:00:27.994130 20477 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:27.994591 20477 heartbeater.cc:511] Master 127.15.139.190:39801 requested a full tablet report, sending...
I20260923 18:00:27.995805 20020 ts_manager.cc:194] Registered new tserver with Master: 782b7da69d0b43b790a187c0770aeb09 (127.15.139.131:41229)
I20260923 18:00:27.996392 20020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:43397
I20260923 18:00:28.006256 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:00:28.016045 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.008465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47376 (local address 127.15.139.190:39801)
0923 18:00:28.008641 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:28.008646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:28.008806 (+ 160us) server_negotiation.cc:407] Connection header received
0923 18:00:28.008906 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:28.008908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:28.008950 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:28.009017 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:28.009720 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.010223 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.011150 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.011354 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.011427 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:28.012897 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:28.012900 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:28.012902 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:28.012926 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:28.014501 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:28.014921 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:28.014924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:28.014925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:28.014996 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:28.015242 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:28.015244 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:28.015245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:28.015454 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:28.015620 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:28.015828 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 18:00:28.015922 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260923 18:00:28.019143 20020 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:00:28.022115 20020 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:28.037827 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.031067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:41229 (local address 127.0.0.1:42650)
0923 18:00:28.032206 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:28.032224 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:28.032329 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:28.032913 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:28.032916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:28.032933 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:28.033135 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:28.033147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.034562 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.034567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:28.036065 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:28.036075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.037119 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.037124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:28.037244 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:28.037286 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 18:00:28.037483 (+ 197us) client_negotiation.cc:240] Negotiation successful
0923 18:00:28.037619 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"mutex_wait_us":52,"thread_start_us":122,"threads_started":1}
I20260923 18:00:28.038107 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.030985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:43217 (local address 127.0.0.1:58896)
0923 18:00:28.031948 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:28.032012 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:28.032168 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:28.032884 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:28.032891 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:28.032914 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:28.033111 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:28.033119 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.035322 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.035330 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:28.036390 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:28.036402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.037666 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.037669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:28.037791 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:28.037813 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 18:00:28.037884 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 18:00:28.037961 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"mutex_wait_us":2,"thread_start_us":108,"threads_started":1}
I20260923 18:00:28.038417 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.032094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52494 (local address 127.15.139.129:35093)
0923 18:00:28.032623 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:28.032632 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:28.033020 (+ 388us) server_negotiation.cc:407] Connection header received
0923 18:00:28.033112 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:28.033118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:28.033335 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:28.033515 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:28.034012 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.035251 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.036603 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.037361 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.037480 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:28.037766 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:28.037820 (+ 54us) server_negotiation.cc:299] Negotiation successful
0923 18:00:28.038160 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":90,"threads_started":1}
I20260923 18:00:28.038584 20491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.031411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58896 (local address 127.15.139.130:43217)
0923 18:00:28.031806 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:28.031819 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:28.032323 (+ 504us) server_negotiation.cc:407] Connection header received
0923 18:00:28.032412 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:28.032418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:28.032684 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:28.032851 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:28.033416 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.035128 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.036553 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.037361 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.037480 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:28.037602 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:28.038298 (+ 696us) server_negotiation.cc:299] Negotiation successful
0923 18:00:28.038406 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":174,"threads_started":1}
I20260923 18:00:28.038829 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.031845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35093 (local address 127.0.0.1:52494)
0923 18:00:28.032756 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:28.032794 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:28.032909 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:28.033701 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:28.033704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:28.033728 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:28.033849 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:28.033857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.035421 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.035424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:28.036456 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:28.036463 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.037510 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.037515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:28.037622 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:28.037649 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 18:00:28.037726 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 18:00:28.037789 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":5,"thread_start_us":106,"threads_started":1}
I20260923 18:00:28.039088 20490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.031258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42650 (local address 127.15.139.131:41229)
0923 18:00:28.032499 (+ 1241us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:28.032509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:28.032528 (+ 19us) server_negotiation.cc:407] Connection header received
0923 18:00:28.032594 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:28.032598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:28.032766 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:28.033098 (+ 332us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:28.033258 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.034424 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.036217 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.036970 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.037136 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:28.037294 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:28.038613 (+ 1319us) server_negotiation.cc:299] Negotiation successful
0923 18:00:28.038875 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1143,"thread_start_us":178,"threads_started":1}
I20260923 18:00:28.041662 20145 tablet_service.cc:1467] Processing CreateTablet for tablet 2562996555e34b349edf7c9859589cda (DEFAULT_TABLE table=test-table [id=d77544a055f64942829ef03d743f7a05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:28.041664 20278 tablet_service.cc:1467] Processing CreateTablet for tablet 2562996555e34b349edf7c9859589cda (DEFAULT_TABLE table=test-table [id=d77544a055f64942829ef03d743f7a05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:28.041664 20411 tablet_service.cc:1467] Processing CreateTablet for tablet 2562996555e34b349edf7c9859589cda (DEFAULT_TABLE table=test-table [id=d77544a055f64942829ef03d743f7a05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:28.042752 20278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2562996555e34b349edf7c9859589cda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:28.042752 20145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2562996555e34b349edf7c9859589cda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:28.042841 20411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2562996555e34b349edf7c9859589cda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:28.049893 20494 tablet_bootstrap.cc:492] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61: Bootstrap starting.
I20260923 18:00:28.050523 20495 tablet_bootstrap.cc:492] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566: Bootstrap starting.
I20260923 18:00:28.051121 20496 tablet_bootstrap.cc:492] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09: Bootstrap starting.
I20260923 18:00:28.053802 20496 tablet_bootstrap.cc:654] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:28.055308 20496 log.cc:824] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:28.055847 20494 tablet_bootstrap.cc:654] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:28.056130 20495 tablet_bootstrap.cc:654] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:28.056684 20494 log.cc:824] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:28.057016 20495 log.cc:824] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:28.057736 20496 tablet_bootstrap.cc:492] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09: No bootstrap required, opened a new log
I20260923 18:00:28.057928 20496 ts_tablet_manager.cc:1397] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260923 18:00:28.058725 20494 tablet_bootstrap.cc:492] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61: No bootstrap required, opened a new log
I20260923 18:00:28.058907 20494 ts_tablet_manager.cc:1397] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260923 18:00:28.058926 20495 tablet_bootstrap.cc:492] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566: No bootstrap required, opened a new log
I20260923 18:00:28.059128 20495 ts_tablet_manager.cc:1397] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260923 18:00:28.061827 20496 raft_consensus.cc:348] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782b7da69d0b43b790a187c0770aeb09" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41229 } } peers { permanent_uuid: "4040195c15ae4da0a5d59dd6be66ab61" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43217 } } peers { permanent_uuid: "0bdd70d5edc74610b55cba728a03d566" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35093 } }
I20260923 18:00:28.062121 20496 raft_consensus.cc:374] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:28.062179 20496 raft_consensus.cc:729] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 782b7da69d0b43b790a187c0770aeb09, State: Initialized, Role: FOLLOWER
I20260923 18:00:28.062638 20496 consensus_queue.cc:260] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782b7da69d0b43b790a187c0770aeb09" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41229 } } peers { permanent_uuid: "4040195c15ae4da0a5d59dd6be66ab61" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43217 } } peers { permanent_uuid: "0bdd70d5edc74610b55cba728a03d566" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35093 } }
I20260923 18:00:28.062619 20494 raft_consensus.cc:348] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782b7da69d0b43b790a187c0770aeb09" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41229 } } peers { permanent_uuid: "4040195c15ae4da0a5d59dd6be66ab61" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43217 } } peers { permanent_uuid: "0bdd70d5edc74610b55cba728a03d566" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35093 } }
I20260923 18:00:28.062870 20494 raft_consensus.cc:374] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:28.062934 20494 raft_consensus.cc:729] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4040195c15ae4da0a5d59dd6be66ab61, State: Initialized, Role: FOLLOWER
I20260923 18:00:28.063367 20494 consensus_queue.cc:260] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782b7da69d0b43b790a187c0770aeb09" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41229 } } peers { permanent_uuid: "4040195c15ae4da0a5d59dd6be66ab61" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43217 } } peers { permanent_uuid: "0bdd70d5edc74610b55cba728a03d566" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35093 } }
I20260923 18:00:28.063699 20496 ts_tablet_manager.cc:1428] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260923 18:00:28.063771 20495 raft_consensus.cc:348] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782b7da69d0b43b790a187c0770aeb09" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41229 } } peers { permanent_uuid: "4040195c15ae4da0a5d59dd6be66ab61" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43217 } } peers { permanent_uuid: "0bdd70d5edc74610b55cba728a03d566" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35093 } }
I20260923 18:00:28.064049 20477 heartbeater.cc:503] Master 127.15.139.190:39801 was elected leader, sending a full tablet report...
I20260923 18:00:28.064052 20495 raft_consensus.cc:374] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:28.064114 20495 raft_consensus.cc:729] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd70d5edc74610b55cba728a03d566, State: Initialized, Role: FOLLOWER
I20260923 18:00:28.064146 20344 heartbeater.cc:503] Master 127.15.139.190:39801 was elected leader, sending a full tablet report...
I20260923 18:00:28.064561 20495 consensus_queue.cc:260] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782b7da69d0b43b790a187c0770aeb09" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41229 } } peers { permanent_uuid: "4040195c15ae4da0a5d59dd6be66ab61" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43217 } } peers { permanent_uuid: "0bdd70d5edc74610b55cba728a03d566" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35093 } }
I20260923 18:00:28.065306 20494 ts_tablet_manager.cc:1428] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260923 18:00:28.066368 20495 ts_tablet_manager.cc:1428] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260923 18:00:28.066778 20211 heartbeater.cc:503] Master 127.15.139.190:39801 was elected leader, sending a full tablet report...
I20260923 18:00:28.200048 20501 raft_consensus.cc:482] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:28.200374 20501 raft_consensus.cc:504] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782b7da69d0b43b790a187c0770aeb09" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41229 } } peers { permanent_uuid: "4040195c15ae4da0a5d59dd6be66ab61" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43217 } } peers { permanent_uuid: "0bdd70d5edc74610b55cba728a03d566" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35093 } }
I20260923 18:00:28.201831 20501 leader_election.cc:288] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 782b7da69d0b43b790a187c0770aeb09 (127.15.139.131:41229), 0bdd70d5edc74610b55cba728a03d566 (127.15.139.129:35093)
I20260923 18:00:28.206403 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.202242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:47597 (local address 127.15.139.129:35093)
0923 18:00:28.202357 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:28.202362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:28.202675 (+ 313us) server_negotiation.cc:407] Connection header received
0923 18:00:28.202742 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:28.202746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:28.202796 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:28.202869 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:28.203138 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.203992 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.205165 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.205782 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.205912 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:28.205963 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:28.206186 (+ 223us) server_negotiation.cc:299] Negotiation successful
0923 18:00:28.206282 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 18:00:28.206415 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.202135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35093 (local address 127.15.139.130:47597)
0923 18:00:28.202586 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:28.202595 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:28.202669 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:28.202885 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:28.202888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:28.202909 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:28.203015 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:28.203022 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.204080 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.204083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:28.205039 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:28.205049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.205966 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.205969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:28.206049 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:28.206070 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 18:00:28.206135 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 18:00:28.206189 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":210,"thread_start_us":103,"threads_started":1}
I20260923 18:00:28.206815 20490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.202373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:43241 (local address 127.15.139.131:41229)
0923 18:00:28.202500 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:28.202505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:28.202520 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:00:28.202600 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:28.202603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:28.202670 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:28.202771 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:28.203100 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.203957 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.205210 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.205806 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.205912 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:28.205963 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:28.206572 (+ 609us) server_negotiation.cc:299] Negotiation successful
0923 18:00:28.206675 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:00:28.206446 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.202136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:41229 (local address 127.15.139.130:43241)
0923 18:00:28.202249 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:28.202265 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:28.202367 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:28.202799 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:28.202801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:28.202814 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:28.202955 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:28.202962 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.204074 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.204078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:28.205046 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:28.205059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.206031 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.206035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:28.206161 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:28.206186 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 18:00:28.206273 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 18:00:28.206338 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":65}
I20260923 18:00:28.207267 20165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2562996555e34b349edf7c9859589cda" candidate_uuid: "4040195c15ae4da0a5d59dd6be66ab61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd70d5edc74610b55cba728a03d566" is_pre_election: true
I20260923 18:00:28.207623 20431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2562996555e34b349edf7c9859589cda" candidate_uuid: "4040195c15ae4da0a5d59dd6be66ab61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "782b7da69d0b43b790a187c0770aeb09" is_pre_election: true
I20260923 18:00:28.207646 20165 raft_consensus.cc:2454] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4040195c15ae4da0a5d59dd6be66ab61 in term 0.
I20260923 18:00:28.207916 20431 raft_consensus.cc:2454] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4040195c15ae4da0a5d59dd6be66ab61 in term 0.
I20260923 18:00:28.208122 20234 leader_election.cc:302] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd70d5edc74610b55cba728a03d566, 4040195c15ae4da0a5d59dd6be66ab61; no voters:
I20260923 18:00:28.208386 20501 raft_consensus.cc:2781] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:28.208469 20501 raft_consensus.cc:482] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:28.208513 20501 raft_consensus.cc:3037] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:28.209578 20501 raft_consensus.cc:504] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782b7da69d0b43b790a187c0770aeb09" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41229 } } peers { permanent_uuid: "4040195c15ae4da0a5d59dd6be66ab61" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43217 } } peers { permanent_uuid: "0bdd70d5edc74610b55cba728a03d566" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35093 } }
I20260923 18:00:28.209897 20501 leader_election.cc:288] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [CANDIDATE]: Term 1 election: Requested vote from peers 782b7da69d0b43b790a187c0770aeb09 (127.15.139.131:41229), 0bdd70d5edc74610b55cba728a03d566 (127.15.139.129:35093)
I20260923 18:00:28.210273 20165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2562996555e34b349edf7c9859589cda" candidate_uuid: "4040195c15ae4da0a5d59dd6be66ab61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd70d5edc74610b55cba728a03d566"
I20260923 18:00:28.210282 20431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2562996555e34b349edf7c9859589cda" candidate_uuid: "4040195c15ae4da0a5d59dd6be66ab61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "782b7da69d0b43b790a187c0770aeb09"
I20260923 18:00:28.210415 20431 raft_consensus.cc:3037] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:28.210386 20165 raft_consensus.cc:3037] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:28.211427 20431 raft_consensus.cc:2454] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4040195c15ae4da0a5d59dd6be66ab61 in term 1.
I20260923 18:00:28.211459 20165 raft_consensus.cc:2454] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4040195c15ae4da0a5d59dd6be66ab61 in term 1.
I20260923 18:00:28.211802 20232 leader_election.cc:302] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4040195c15ae4da0a5d59dd6be66ab61, 782b7da69d0b43b790a187c0770aeb09; no voters:
I20260923 18:00:28.211999 20501 raft_consensus.cc:2781] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:28.212205 20501 raft_consensus.cc:686] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 1 LEADER]: Becoming Leader. State: Replica: 4040195c15ae4da0a5d59dd6be66ab61, State: Running, Role: LEADER
I20260923 18:00:28.212539 20501 consensus_queue.cc:237] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782b7da69d0b43b790a187c0770aeb09" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41229 } } peers { permanent_uuid: "4040195c15ae4da0a5d59dd6be66ab61" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43217 } } peers { permanent_uuid: "0bdd70d5edc74610b55cba728a03d566" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35093 } }
I20260923 18:00:28.215832 20019 catalog_manager.cc:5495] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4040195c15ae4da0a5d59dd6be66ab61 (127.15.139.130). New cstate: current_term: 1 leader_uuid: "4040195c15ae4da0a5d59dd6be66ab61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782b7da69d0b43b790a187c0770aeb09" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4040195c15ae4da0a5d59dd6be66ab61" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd70d5edc74610b55cba728a03d566" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35093 } health_report { overall_health: UNKNOWN } } }
I20260923 18:00:28.227311 20491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.223239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58898 (local address 127.15.139.130:43217)
0923 18:00:28.223372 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:28.223376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:28.223393 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:00:28.223469 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:28.223471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:28.223593 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:28.223700 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:28.224141 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.224640 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.225605 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:28.225820 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:28.225905 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:28.226228 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:28.226731 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:28.226824 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:28.227087 (+ 263us) server_negotiation.cc:299] Negotiation successful
0923 18:00:28.227141 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260923 18:00:28.227928 20478 tablet.cc:2369] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:00:28.233345 20431 raft_consensus.cc:1261] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09 [term 1 FOLLOWER]: Refusing update from remote peer 4040195c15ae4da0a5d59dd6be66ab61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:28.233345 20165 raft_consensus.cc:1261] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566 [term 1 FOLLOWER]: Refusing update from remote peer 4040195c15ae4da0a5d59dd6be66ab61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:28.234129 20504 consensus_queue.cc:1035] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "782b7da69d0b43b790a187c0770aeb09" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:28.234309 20501 consensus_queue.cc:1035] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd70d5edc74610b55cba728a03d566" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:28.242764 20507 mvcc.cc:205] Tried to move back new op lower bound from 7332603610035888128 to 7332603609961484288. Current Snapshot: MvccSnapshot[applied={T|T < 7332603610035888128}]
I20260923 18:00:28.244161 20509 mvcc.cc:205] Tried to move back new op lower bound from 7332603610035888128 to 7332603609961484288. Current Snapshot: MvccSnapshot[applied={T|T < 7332603610035888128}]
I20260923 18:00:28.244272 20508 mvcc.cc:205] Tried to move back new op lower bound from 7332603610035888128 to 7332603609961484288. Current Snapshot: MvccSnapshot[applied={T|T < 7332603610035888128}]
I20260923 18:00:28.252199 20021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:28.252389 20021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:28.255203 20021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1057ce2c604b4637957ca8a99c2cca19: Sending DeleteTablet for 3 replicas of tablet 2562996555e34b349edf7c9859589cda
I20260923 18:00:28.256243 20278 tablet_service.cc:1514] Processing DeleteTablet for tablet 2562996555e34b349edf7c9859589cda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:28 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:58896
I20260923 18:00:28.256310 20145 tablet_service.cc:1514] Processing DeleteTablet for tablet 2562996555e34b349edf7c9859589cda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:28 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:52494
I20260923 18:00:28.256498 20411 tablet_service.cc:1514] Processing DeleteTablet for tablet 2562996555e34b349edf7c9859589cda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:28 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:42650
I20260923 18:00:28.256966 20519 tablet_replica.cc:330] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09: stopping tablet replica
I20260923 18:00:28.257380 20517 tablet_replica.cc:330] T 2562996555e34b349edf7c9859589cda P 0bdd70d5edc74610b55cba728a03d566: stopping tablet replica
I20260923 18:00:28.257313 20519 raft_consensus.cc:2229] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:28.257795 20519 raft_consensus.cc:2258] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:28.257915 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 20083
I20260923 18:00:28.259974 20518 tablet_replica.cc:330] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61: stopping tablet replica
I20260923 18:00:28.260299 20518 raft_consensus.cc:2229] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:28.260715 20518 raft_consensus.cc:2258] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:28.262393 20519 ts_tablet_manager.cc:1905] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:28.262833 20518 ts_tablet_manager.cc:1905] T 2562996555e34b349edf7c9859589cda P 4040195c15ae4da0a5d59dd6be66ab61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:28.265502 20519 ts_tablet_manager.cc:1918] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:28.265614 20519 log.cc:1196] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/2562996555e34b349edf7c9859589cda
I20260923 18:00:28.265985 20519 ts_tablet_manager.cc:1939] T 2562996555e34b349edf7c9859589cda P 782b7da69d0b43b790a187c0770aeb09: Deleting consensus metadata
W20260923 18:00:28.267009 20008 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:35093 (error 108)
I20260923 18:00:28.267699 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 20216
I20260923 18:00:28.267706 20006 catalog_manager.cc:4841] TS 782b7da69d0b43b790a187c0770aeb09 (127.15.139.131:41229): tablet 2562996555e34b349edf7c9859589cda (table test-table [id=d77544a055f64942829ef03d743f7a05]) successfully deleted
I20260923 18:00:28.270051 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.269655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35093 (local address 127.0.0.1:52508)
0923 18:00:28.269798 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:28.269866 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:35093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260923 18:00:28.270272 20008 catalog_manager.cc:4584] TS 0bdd70d5edc74610b55cba728a03d566 (127.15.139.129:35093): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2562996555e34b349edf7c9859589cda: Network error: Client connection negotiation failed: client connection to 127.15.139.129:35093: connect: Connection refused (error 111)
I20260923 18:00:28.274780 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 20349
I20260923 18:00:28.276772 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.275027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:43217 (local address 127.0.0.1:58910)
0923 18:00:28.276587 (+ 1560us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:28.276665 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.130:43217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1434}
I20260923 18:00:28.281097 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 19990
2026-09-23T18:00:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2414 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-23T18:00:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:28Z Disabled control of system clock
I20260923 18:00:28.313495 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:42429
--webserver_interface=127.15.139.190
--webserver_port=0
--builtin_ntp_servers=127.15.139.148:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:42429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 18:00:28.427361 20524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:28.427958 20524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:28.428045 20524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:28.432152 20524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:28.432224 20524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:28.432242 20524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:28.432262 20524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:28.432325 20524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:28.437392 20524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:42429
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:42429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:28.438566 20524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:28.440145 20524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:28.449203 20532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:28.449208 20529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:28.449331 20524 server_base.cc:1034] running on GCE node
W20260923 18:00:28.449272 20530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:28.450788 20524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:28.452114 20524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:28.453332 20524 hybrid_clock.cc:648] HybridClock initialized: now 1790186428453290 us; error 70 us; skew 500 ppm
I20260923 18:00:28.455282 20524 webserver.cc:460] Webserver started at http://127.15.139.190:33831/ using document root <none> and password file <none>
I20260923 18:00:28.455951 20524 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:28.456048 20524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:28.456282 20524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:28.458359 20524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "8b110dc8b39b4028b3dfda2ae7784639"
format_stamp: "Formatted at 2026-09-23 18:00:28 on dist-test-slave-vr57"
server_key: "7fae12b1b28c5b0db207696ab310e784"
server_key_iv: "9eb7e5fb51c7a5d4937094c4f534f901"
server_key_version: "encryptionkey@0"
I20260923 18:00:28.458992 20524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "8b110dc8b39b4028b3dfda2ae7784639"
format_stamp: "Formatted at 2026-09-23 18:00:28 on dist-test-slave-vr57"
server_key: "7fae12b1b28c5b0db207696ab310e784"
server_key_iv: "9eb7e5fb51c7a5d4937094c4f534f901"
server_key_version: "encryptionkey@0"
I20260923 18:00:28.462702 20524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260923 18:00:28.465148 20538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:28.466362 20524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:28.466500 20524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "8b110dc8b39b4028b3dfda2ae7784639"
format_stamp: "Formatted at 2026-09-23 18:00:28 on dist-test-slave-vr57"
server_key: "7fae12b1b28c5b0db207696ab310e784"
server_key_iv: "9eb7e5fb51c7a5d4937094c4f534f901"
server_key_version: "encryptionkey@0"
I20260923 18:00:28.466655 20524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:28.471328 20524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:28.471977 20524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:28.472169 20524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:28.478839 20524 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:42429
I20260923 18:00:28.478850 20590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:42429 every 8 connection(s)
I20260923 18:00:28.479844 20524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:28.483018 20591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:28.488610 20591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639: Bootstrap starting.
I20260923 18:00:28.489671 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 20524
I20260923 18:00:28.489825 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:28.490160 15918 external_mini_cluster.cc:1428] Setting key 5584389b98a67127982d4340993acdae
I20260923 18:00:28.491899 20591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:28.492897 20591 log.cc:824] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:28.495136 20591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639: No bootstrap required, opened a new log
I20260923 18:00:28.496815 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.491859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48808 (local address 127.15.139.190:42429)
0923 18:00:28.492747 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:28.492771 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:28.492815 (+ 44us) server_negotiation.cc:407] Connection header received
0923 18:00:28.493596 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:28.493616 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:28.494011 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:28.494381 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:28.495074 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:28.495079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:28.495094 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:28.495105 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:28.495175 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:28.495201 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:28.495307 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:28.495453 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 18:00:28.495724 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":144,"threads_started":1}
I20260923 18:00:28.499228 20591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b110dc8b39b4028b3dfda2ae7784639" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42429 } }
I20260923 18:00:28.499449 20591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:28.499538 20591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b110dc8b39b4028b3dfda2ae7784639, State: Initialized, Role: FOLLOWER
I20260923 18:00:28.499933 20591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b110dc8b39b4028b3dfda2ae7784639" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42429 } }
I20260923 18:00:28.500052 20591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:28.500097 20591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:28.500165 20591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:28.501051 20591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b110dc8b39b4028b3dfda2ae7784639" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42429 } }
I20260923 18:00:28.501338 20591 leader_election.cc:302] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b110dc8b39b4028b3dfda2ae7784639; no voters:
I20260923 18:00:28.501636 20591 leader_election.cc:288] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:28.501709 20596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:28.501962 20596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [term 1 LEADER]: Becoming Leader. State: Replica: 8b110dc8b39b4028b3dfda2ae7784639, State: Running, Role: LEADER
I20260923 18:00:28.502275 20596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b110dc8b39b4028b3dfda2ae7784639" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42429 } }
I20260923 18:00:28.502897 20591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:28.503842 20597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b110dc8b39b4028b3dfda2ae7784639" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b110dc8b39b4028b3dfda2ae7784639" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42429 } } }
I20260923 18:00:28.504007 20597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:28.504377 20605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:28.504344 20598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b110dc8b39b4028b3dfda2ae7784639. Latest consensus state: current_term: 1 leader_uuid: "8b110dc8b39b4028b3dfda2ae7784639" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b110dc8b39b4028b3dfda2ae7784639" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42429 } } }
I20260923 18:00:28.504463 20598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:28.507730 20605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:28.513477 20605 catalog_manager.cc:1347] Generated new cluster ID: 57aec0740c7741ebb6f4eb8db6013dd7
I20260923 18:00:28.513554 20605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:28.522931 20605 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:28.523270 20605 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:28.529918 20605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639: Generated new TSK 0
I20260923 18:00:28.530900 20605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:00:28.539184 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42429
--builtin_ntp_servers=127.15.139.148:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 18:00:28.654198 20615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:28.654476 20615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:28.654518 20615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:28.658648 20615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:28.658706 20615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:28.658763 20615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:28.664188 20615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:28.665329 20615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:28.666812 20615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:28.677186 20621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:28.677186 20623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:28.677205 20620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:28.677413 20615 server_base.cc:1034] running on GCE node
I20260923 18:00:28.678118 20615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:28.678953 20615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:28.680153 20615 hybrid_clock.cc:648] HybridClock initialized: now 1790186428680138 us; error 27 us; skew 500 ppm
I20260923 18:00:28.682107 20615 webserver.cc:460] Webserver started at http://127.15.139.129:43403/ using document root <none> and password file <none>
I20260923 18:00:28.682751 20615 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:28.682852 20615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:28.683136 20615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:28.685333 20615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "fff18e9c125e4f018e54c174e78ead29"
format_stamp: "Formatted at 2026-09-23 18:00:28 on dist-test-slave-vr57"
server_key: "542582760a02dbd78113bb6fa12e8f28"
server_key_iv: "83a9c277bbc6c338cac2953ae1fac399"
server_key_version: "encryptionkey@0"
I20260923 18:00:28.685935 20615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "fff18e9c125e4f018e54c174e78ead29"
format_stamp: "Formatted at 2026-09-23 18:00:28 on dist-test-slave-vr57"
server_key: "542582760a02dbd78113bb6fa12e8f28"
server_key_iv: "83a9c277bbc6c338cac2953ae1fac399"
server_key_version: "encryptionkey@0"
I20260923 18:00:28.689730 20615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:28.692535 20629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:28.693967 20615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:28.694136 20615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "fff18e9c125e4f018e54c174e78ead29"
format_stamp: "Formatted at 2026-09-23 18:00:28 on dist-test-slave-vr57"
server_key: "542582760a02dbd78113bb6fa12e8f28"
server_key_iv: "83a9c277bbc6c338cac2953ae1fac399"
server_key_version: "encryptionkey@0"
I20260923 18:00:28.694278 20615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:28.699159 20615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:28.699842 20615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:28.700040 20615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:28.700582 20615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:28.701507 20615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:28.701577 20615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:28.701645 20615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:28.701696 20615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:28.711230 20615 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:40679
I20260923 18:00:28.711297 20742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:40679 every 8 connection(s)
I20260923 18:00:28.712268 20615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:28.715813 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 20615
I20260923 18:00:28.715945 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:28.716272 15918 external_mini_cluster.cc:1428] Setting key 7e0fa85c2028f1fdab3991458b04a502
I20260923 18:00:28.718756 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42429
--builtin_ntp_servers=127.15.139.148:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 18:00:28.719254 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.714131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:34963 (local address 127.15.139.190:42429)
0923 18:00:28.714306 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:28.714310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:28.715550 (+ 1240us) server_negotiation.cc:407] Connection header received
0923 18:00:28.717350 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:28.717352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:28.717397 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:28.717482 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:28.718737 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:28.718739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:28.718741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:28.718753 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:28.718771 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:28.718779 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:28.718892 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:28.719068 (+ 176us) server_negotiation.cc:299] Negotiation successful
0923 18:00:28.719114 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 18:00:28.720222 20745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.714648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42429 (local address 127.15.139.129:34963)
0923 18:00:28.715334 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:28.715376 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:28.717088 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:28.717692 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:28.717702 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:28.717818 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:28.718414 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:00:28.718417 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:28.718473 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:00:28.718570 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:28.718881 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:28.718891 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:28.718898 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 18:00:28.719115 (+ 217us) client_negotiation.cc:240] Negotiation successful
0923 18:00:28.719368 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":110,"threads_started":1}
I20260923 18:00:28.721580 20743 heartbeater.cc:347] Connected to a master server at 127.15.139.190:42429
I20260923 18:00:28.721822 20743 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:28.722254 20743 heartbeater.cc:511] Master 127.15.139.190:42429 requested a full tablet report, sending...
I20260923 18:00:28.723982 20555 ts_manager.cc:194] Registered new tserver with Master: fff18e9c125e4f018e54c174e78ead29 (127.15.139.129:40679)
W20260923 18:00:28.834702 20746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:28.834975 20746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:28.835016 20746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:28.839200 20746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:28.839262 20746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:28.839319 20746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:28.844115 20746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:28.845328 20746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:28.846801 20746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:28.857524 20751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:28.857570 20752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:28.857524 20754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:28.857811 20746 server_base.cc:1034] running on GCE node
I20260923 18:00:28.858232 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:28.858915 20746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:28.860105 20746 hybrid_clock.cc:648] HybridClock initialized: now 1790186428860088 us; error 39 us; skew 500 ppm
I20260923 18:00:28.862041 20746 webserver.cc:460] Webserver started at http://127.15.139.130:44289/ using document root <none> and password file <none>
I20260923 18:00:28.862643 20746 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:28.862739 20746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:28.862972 20746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:28.865136 20746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3"
format_stamp: "Formatted at 2026-09-23 18:00:28 on dist-test-slave-vr57"
server_key: "a9b6006f528b3ffacea66b799d516539"
server_key_iv: "d015ff731cc37f9b08f0e279b679ecca"
server_key_version: "encryptionkey@0"
I20260923 18:00:28.865720 20746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3"
format_stamp: "Formatted at 2026-09-23 18:00:28 on dist-test-slave-vr57"
server_key: "a9b6006f528b3ffacea66b799d516539"
server_key_iv: "d015ff731cc37f9b08f0e279b679ecca"
server_key_version: "encryptionkey@0"
I20260923 18:00:28.869359 20746 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260923 18:00:28.872078 20760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:28.873297 20746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:00:28.873428 20746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3"
format_stamp: "Formatted at 2026-09-23 18:00:28 on dist-test-slave-vr57"
server_key: "a9b6006f528b3ffacea66b799d516539"
server_key_iv: "d015ff731cc37f9b08f0e279b679ecca"
server_key_version: "encryptionkey@0"
I20260923 18:00:28.873574 20746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:28.878331 20746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:28.878954 20746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:28.879141 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:28.879738 20746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:28.880677 20746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:28.880748 20746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:28.880816 20746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:28.880867 20746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:28.890734 20746 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:39369
I20260923 18:00:28.890751 20873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:39369 every 8 connection(s)
I20260923 18:00:28.891793 20746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:28.895848 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 20746
I20260923 18:00:28.895985 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:28.896279 15918 external_mini_cluster.cc:1428] Setting key 839c2a4578a115d0e48c4153b77b4f13
I20260923 18:00:28.898330 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42429
--builtin_ntp_servers=127.15.139.148:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 18:00:28.898674 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.893921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:52641 (local address 127.15.139.190:42429)
0923 18:00:28.894069 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:28.894072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:28.895195 (+ 1123us) server_negotiation.cc:407] Connection header received
0923 18:00:28.896751 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:28.896754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:28.896800 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:28.896919 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:28.898171 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:28.898174 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:28.898177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:28.898190 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:28.898218 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:28.898226 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:28.898320 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:28.898490 (+ 170us) server_negotiation.cc:299] Negotiation successful
0923 18:00:28.898535 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 18:00:28.899793 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:28.894353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42429 (local address 127.15.139.130:52641)
0923 18:00:28.895040 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:28.895083 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:28.896530 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:28.897096 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:28.897103 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:28.897200 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:28.897863 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:00:28.897864 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:28.897920 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:00:28.898018 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:28.898307 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:28.898316 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:28.898322 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 18:00:28.898538 (+ 216us) client_negotiation.cc:240] Negotiation successful
0923 18:00:28.898879 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":125,"threads_started":1}
I20260923 18:00:28.901017 20874 heartbeater.cc:347] Connected to a master server at 127.15.139.190:42429
I20260923 18:00:28.901262 20874 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:28.901774 20874 heartbeater.cc:511] Master 127.15.139.190:42429 requested a full tablet report, sending...
I20260923 18:00:28.902952 20555 ts_manager.cc:194] Registered new tserver with Master: feca9b7cf6634a13aaa65a1b9c5e54f3 (127.15.139.130:39369)
W20260923 18:00:29.016371 20877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:29.016705 20877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:29.016798 20877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:29.021165 20877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:29.021271 20877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:29.021370 20877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:29.027112 20877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:29.028389 20877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:29.030028 20877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:29.040143 20883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:29.040386 20877 server_base.cc:1034] running on GCE node
W20260923 18:00:29.040138 20882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:29.040138 20885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:29.041072 20877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:29.041798 20877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:29.043035 20877 hybrid_clock.cc:648] HybridClock initialized: now 1790186429043013 us; error 34 us; skew 500 ppm
I20260923 18:00:29.045224 20877 webserver.cc:460] Webserver started at http://127.15.139.131:36647/ using document root <none> and password file <none>
I20260923 18:00:29.045849 20877 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:29.045960 20877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:29.046211 20877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:29.048907 20877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "397270dade274d91949169fdb488c29b"
format_stamp: "Formatted at 2026-09-23 18:00:29 on dist-test-slave-vr57"
server_key: "41ef987a3d9b91023b5baf87b844650a"
server_key_iv: "24e81f96a8341e19e73d9788cacd3694"
server_key_version: "encryptionkey@0"
I20260923 18:00:29.049520 20877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "397270dade274d91949169fdb488c29b"
format_stamp: "Formatted at 2026-09-23 18:00:29 on dist-test-slave-vr57"
server_key: "41ef987a3d9b91023b5baf87b844650a"
server_key_iv: "24e81f96a8341e19e73d9788cacd3694"
server_key_version: "encryptionkey@0"
I20260923 18:00:29.053851 20877 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260923 18:00:29.056607 20891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:29.057876 20877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:00:29.058069 20877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "397270dade274d91949169fdb488c29b"
format_stamp: "Formatted at 2026-09-23 18:00:29 on dist-test-slave-vr57"
server_key: "41ef987a3d9b91023b5baf87b844650a"
server_key_iv: "24e81f96a8341e19e73d9788cacd3694"
server_key_version: "encryptionkey@0"
I20260923 18:00:29.058200 20877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:29.063175 20877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:29.063856 20877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:29.064061 20877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:29.064637 20877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:29.065635 20877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:29.065711 20877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:29.065788 20877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:29.065845 20877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:29.075806 20877 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:44915
I20260923 18:00:29.075830 21004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:44915 every 8 connection(s)
I20260923 18:00:29.076843 20877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:29.083310 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.078898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:42037 (local address 127.15.139.190:42429)
0923 18:00:29.079055 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:29.079060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:29.080327 (+ 1267us) server_negotiation.cc:407] Connection header received
0923 18:00:29.081670 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:29.081673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:29.081713 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:29.081800 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:29.082889 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:29.082892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:29.082895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:29.082907 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:29.082927 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:29.082938 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:29.083036 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:29.083160 (+ 124us) server_negotiation.cc:299] Negotiation successful
0923 18:00:29.083199 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 18:00:29.084183 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.079401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42429 (local address 127.15.139.131:42037)
0923 18:00:29.080159 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:29.080200 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:29.081460 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:29.081954 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:29.081961 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:29.082075 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:29.082641 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:00:29.082642 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:29.082681 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:00:29.082746 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:29.083018 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:29.083022 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:29.083025 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 18:00:29.083176 (+ 151us) client_negotiation.cc:240] Negotiation successful
0923 18:00:29.083389 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":121,"threads_started":1}
I20260923 18:00:29.085357 21005 heartbeater.cc:347] Connected to a master server at 127.15.139.190:42429
I20260923 18:00:29.085542 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 20877
I20260923 18:00:29.085602 21005 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:29.085671 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:00:29.085973 15918 external_mini_cluster.cc:1428] Setting key 6bc5b25017b1bb28117185ad926e4f20
I20260923 18:00:29.086078 21005 heartbeater.cc:511] Master 127.15.139.190:42429 requested a full tablet report, sending...
I20260923 18:00:29.087255 20555 ts_manager.cc:194] Registered new tserver with Master: 397270dade274d91949169fdb488c29b (127.15.139.131:44915)
I20260923 18:00:29.088320 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:00:29.092275 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.090323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48824 (local address 127.15.139.190:42429)
0923 18:00:29.090537 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:29.090541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:29.090685 (+ 144us) server_negotiation.cc:407] Connection header received
0923 18:00:29.090846 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:29.090849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:29.090895 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:29.091041 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:29.091860 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:29.091864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:29.091867 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:29.091878 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:29.091908 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:29.091923 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:29.092016 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:29.092097 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 18:00:29.092134 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260923 18:00:29.095304 20554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:00:29.098459 20554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:29.110634 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.107281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:40679 (local address 127.0.0.1:32866)
0923 18:00:29.108403 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:29.108461 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:29.108629 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:29.109102 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:29.109108 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:29.109184 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:29.109769 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:00:29.109771 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:29.109792 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:00:29.109863 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:29.110230 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:29.110241 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:29.110249 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 18:00:29.110368 (+ 119us) client_negotiation.cc:240] Negotiation successful
0923 18:00:29.110442 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"thread_start_us":88,"threads_started":1}
I20260923 18:00:29.111075 21017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.107432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.15.139.129:40679)
0923 18:00:29.107909 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:29.107924 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:29.108595 (+ 671us) server_negotiation.cc:407] Connection header received
0923 18:00:29.108740 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:29.108747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:29.108961 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:29.109115 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:29.110019 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:29.110025 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:29.110038 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:29.110051 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:29.110096 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:29.110132 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:29.110253 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:29.110831 (+ 578us) server_negotiation.cc:299] Negotiation successful
0923 18:00:29.110910 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":157,"threads_started":1}
I20260923 18:00:29.111230 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.107423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:44915 (local address 127.0.0.1:57796)
0923 18:00:29.108803 (+ 1380us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:29.108824 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:29.108929 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:29.109755 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:29.109760 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:29.109774 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:29.110542 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:00:29.110544 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:29.110555 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:00:29.110582 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:29.110954 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:29.110957 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:29.110961 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 18:00:29.111034 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 18:00:29.111076 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1321,"thread_start_us":639,"threads_started":1}
I20260923 18:00:29.111639 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.107999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:39369 (local address 127.0.0.1:48714)
0923 18:00:29.108831 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:29.108843 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:29.108915 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:29.109986 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:29.109990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:29.110005 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:29.110542 (+ 537us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:00:29.110544 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:29.110557 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:00:29.110582 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:29.111252 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:29.111255 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:29.111259 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 18:00:29.111338 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 18:00:29.111371 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":110,"threads_started":1}
I20260923 18:00:29.111917 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.107761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57796 (local address 127.15.139.131:44915)
0923 18:00:29.108127 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:29.108140 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:29.108930 (+ 790us) server_negotiation.cc:407] Connection header received
0923 18:00:29.109018 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:29.109024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:29.109221 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:29.109359 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:29.110723 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:29.110729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:29.110740 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:29.110751 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:29.110793 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:29.110823 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:29.110914 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:29.111319 (+ 405us) server_negotiation.cc:299] Negotiation successful
0923 18:00:29.111643 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":145,"threads_started":1}
I20260923 18:00:29.112047 21021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.108402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48714 (local address 127.15.139.130:39369)
0923 18:00:29.108722 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:29.108733 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:29.109012 (+ 279us) server_negotiation.cc:407] Connection header received
0923 18:00:29.109073 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:29.109079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:29.109251 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:29.109370 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:29.110906 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:29.110916 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:29.110930 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:29.110944 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:29.110986 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:29.111023 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:29.111114 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:29.111819 (+ 705us) server_negotiation.cc:299] Negotiation successful
0923 18:00:29.111888 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":138,"threads_started":1}
I20260923 18:00:29.114563 20939 tablet_service.cc:1467] Processing CreateTablet for tablet efcd8ed626674dd8b56490dff64d9b92 (DEFAULT_TABLE table=test-table [id=c6d0f1e14b534ba9a280d86268969d81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:29.114558 20659 tablet_service.cc:1467] Processing CreateTablet for tablet efcd8ed626674dd8b56490dff64d9b92 (DEFAULT_TABLE table=test-table [id=c6d0f1e14b534ba9a280d86268969d81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:29.115765 20939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efcd8ed626674dd8b56490dff64d9b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:29.115767 20659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efcd8ed626674dd8b56490dff64d9b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:29.117261 20808 tablet_service.cc:1467] Processing CreateTablet for tablet efcd8ed626674dd8b56490dff64d9b92 (DEFAULT_TABLE table=test-table [id=c6d0f1e14b534ba9a280d86268969d81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:29.118325 20808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efcd8ed626674dd8b56490dff64d9b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:29.123394 21022 tablet_bootstrap.cc:492] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b: Bootstrap starting.
I20260923 18:00:29.124174 21023 tablet_bootstrap.cc:492] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29: Bootstrap starting.
I20260923 18:00:29.125099 21024 tablet_bootstrap.cc:492] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3: Bootstrap starting.
I20260923 18:00:29.126745 21023 tablet_bootstrap.cc:654] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:29.127244 21024 tablet_bootstrap.cc:654] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:29.127724 21023 log.cc:824] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:29.128108 21024 log.cc:824] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:29.129105 21022 tablet_bootstrap.cc:654] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:29.129930 21022 log.cc:824] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:29.129972 21023 tablet_bootstrap.cc:492] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29: No bootstrap required, opened a new log
I20260923 18:00:29.130149 21024 tablet_bootstrap.cc:492] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3: No bootstrap required, opened a new log
I20260923 18:00:29.130188 21023 ts_tablet_manager.cc:1397] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260923 18:00:29.130357 21024 ts_tablet_manager.cc:1397] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260923 18:00:29.132079 21022 tablet_bootstrap.cc:492] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b: No bootstrap required, opened a new log
I20260923 18:00:29.132289 21022 ts_tablet_manager.cc:1397] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.001s
I20260923 18:00:29.134011 21024 raft_consensus.cc:348] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff18e9c125e4f018e54c174e78ead29" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 40679 } } peers { permanent_uuid: "397270dade274d91949169fdb488c29b" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 44915 } } peers { permanent_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39369 } }
I20260923 18:00:29.134296 21024 raft_consensus.cc:374] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:29.134351 21024 raft_consensus.cc:729] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feca9b7cf6634a13aaa65a1b9c5e54f3, State: Initialized, Role: FOLLOWER
I20260923 18:00:29.134814 21024 consensus_queue.cc:260] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff18e9c125e4f018e54c174e78ead29" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 40679 } } peers { permanent_uuid: "397270dade274d91949169fdb488c29b" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 44915 } } peers { permanent_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39369 } }
I20260923 18:00:29.134838 21023 raft_consensus.cc:348] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff18e9c125e4f018e54c174e78ead29" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 40679 } } peers { permanent_uuid: "397270dade274d91949169fdb488c29b" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 44915 } } peers { permanent_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39369 } }
I20260923 18:00:29.135104 21023 raft_consensus.cc:374] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:29.135154 21023 raft_consensus.cc:729] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fff18e9c125e4f018e54c174e78ead29, State: Initialized, Role: FOLLOWER
I20260923 18:00:29.135746 21023 consensus_queue.cc:260] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff18e9c125e4f018e54c174e78ead29" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 40679 } } peers { permanent_uuid: "397270dade274d91949169fdb488c29b" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 44915 } } peers { permanent_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39369 } }
I20260923 18:00:29.135977 21024 ts_tablet_manager.cc:1428] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260923 18:00:29.136423 20874 heartbeater.cc:503] Master 127.15.139.190:42429 was elected leader, sending a full tablet report...
I20260923 18:00:29.136746 20743 heartbeater.cc:503] Master 127.15.139.190:42429 was elected leader, sending a full tablet report...
I20260923 18:00:29.137225 21022 raft_consensus.cc:348] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff18e9c125e4f018e54c174e78ead29" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 40679 } } peers { permanent_uuid: "397270dade274d91949169fdb488c29b" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 44915 } } peers { permanent_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39369 } }
I20260923 18:00:29.137547 21022 raft_consensus.cc:374] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:29.137614 21022 raft_consensus.cc:729] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 397270dade274d91949169fdb488c29b, State: Initialized, Role: FOLLOWER
I20260923 18:00:29.138116 21022 consensus_queue.cc:260] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff18e9c125e4f018e54c174e78ead29" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 40679 } } peers { permanent_uuid: "397270dade274d91949169fdb488c29b" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 44915 } } peers { permanent_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39369 } }
I20260923 18:00:29.138751 21023 ts_tablet_manager.cc:1428] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260923 18:00:29.139093 21022 ts_tablet_manager.cc:1428] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b: Time spent starting tablet: real 0.007s user 0.000s sys 0.007s
I20260923 18:00:29.139286 21005 heartbeater.cc:503] Master 127.15.139.190:42429 was elected leader, sending a full tablet report...
W20260923 18:00:29.142720 20875 tablet.cc:2369] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:00:29.213512 20744 tablet.cc:2369] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:00:29.328025 21006 tablet.cc:2369] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:00:29.356751 21028 raft_consensus.cc:482] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:29.357038 21028 raft_consensus.cc:504] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff18e9c125e4f018e54c174e78ead29" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 40679 } } peers { permanent_uuid: "397270dade274d91949169fdb488c29b" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 44915 } } peers { permanent_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39369 } }
I20260923 18:00:29.358273 21028 leader_election.cc:288] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fff18e9c125e4f018e54c174e78ead29 (127.15.139.129:40679), 397270dade274d91949169fdb488c29b (127.15.139.131:44915)
I20260923 18:00:29.360575 21017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.358678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:33771 (local address 127.15.139.129:40679)
0923 18:00:29.358788 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:29.358806 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:29.359136 (+ 330us) server_negotiation.cc:407] Connection header received
0923 18:00:29.359280 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:29.359283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:29.359330 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:29.359421 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:29.360176 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:29.360180 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:29.360183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:29.360197 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:29.360222 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:29.360231 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:29.360329 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:29.360395 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 18:00:29.360440 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 18:00:29.360589 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.358936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:52047 (local address 127.15.139.131:44915)
0923 18:00:29.359088 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:29.359092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:29.359106 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:00:29.359148 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:29.359150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:29.359180 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:29.359287 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:29.360099 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:29.360103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:29.360106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:29.360120 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:29.360149 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:29.360159 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:29.360271 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:29.360398 (+ 127us) server_negotiation.cc:299] Negotiation successful
0923 18:00:29.360451 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 18:00:29.360589 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.358590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:44915 (local address 127.15.139.130:52047)
0923 18:00:29.358760 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:29.358778 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:29.358878 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:29.359308 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:29.359312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:29.359326 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:29.359845 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:00:29.359847 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:29.359857 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:00:29.359883 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:29.360254 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:29.360266 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:29.360270 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 18:00:29.360394 (+ 124us) client_negotiation.cc:240] Negotiation successful
0923 18:00:29.360444 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":3}
I20260923 18:00:29.360596 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.358609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:40679 (local address 127.15.139.130:33771)
0923 18:00:29.358996 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:29.359015 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:29.359161 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:29.359431 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:29.359433 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:29.359449 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:29.360005 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:00:29.360006 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:29.360016 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:00:29.360039 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:29.360310 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:29.360313 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:29.360316 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 18:00:29.360393 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 18:00:29.360435 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":118,"threads_started":1}
I20260923 18:00:29.361296 20959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efcd8ed626674dd8b56490dff64d9b92" candidate_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "397270dade274d91949169fdb488c29b" is_pre_election: true
I20260923 18:00:29.361421 20697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efcd8ed626674dd8b56490dff64d9b92" candidate_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fff18e9c125e4f018e54c174e78ead29" is_pre_election: true
I20260923 18:00:29.361639 20959 raft_consensus.cc:2454] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate feca9b7cf6634a13aaa65a1b9c5e54f3 in term 0.
I20260923 18:00:29.361835 20697 raft_consensus.cc:2454] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate feca9b7cf6634a13aaa65a1b9c5e54f3 in term 0.
I20260923 18:00:29.362180 20764 leader_election.cc:302] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 397270dade274d91949169fdb488c29b, feca9b7cf6634a13aaa65a1b9c5e54f3; no voters:
I20260923 18:00:29.362522 21028 raft_consensus.cc:2781] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:29.362610 21028 raft_consensus.cc:482] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:29.362670 21028 raft_consensus.cc:3037] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:29.363865 21028 raft_consensus.cc:504] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff18e9c125e4f018e54c174e78ead29" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 40679 } } peers { permanent_uuid: "397270dade274d91949169fdb488c29b" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 44915 } } peers { permanent_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39369 } }
I20260923 18:00:29.364202 21028 leader_election.cc:288] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [CANDIDATE]: Term 1 election: Requested vote from peers fff18e9c125e4f018e54c174e78ead29 (127.15.139.129:40679), 397270dade274d91949169fdb488c29b (127.15.139.131:44915)
I20260923 18:00:29.364543 20959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efcd8ed626674dd8b56490dff64d9b92" candidate_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "397270dade274d91949169fdb488c29b"
I20260923 18:00:29.364589 20697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efcd8ed626674dd8b56490dff64d9b92" candidate_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fff18e9c125e4f018e54c174e78ead29"
I20260923 18:00:29.364698 20697 raft_consensus.cc:3037] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:29.364688 20959 raft_consensus.cc:3037] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:29.366065 20697 raft_consensus.cc:2454] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate feca9b7cf6634a13aaa65a1b9c5e54f3 in term 1.
I20260923 18:00:29.366065 20959 raft_consensus.cc:2454] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate feca9b7cf6634a13aaa65a1b9c5e54f3 in term 1.
I20260923 18:00:29.366395 20764 leader_election.cc:302] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 397270dade274d91949169fdb488c29b, feca9b7cf6634a13aaa65a1b9c5e54f3; no voters:
I20260923 18:00:29.366590 21028 raft_consensus.cc:2781] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:29.366832 21028 raft_consensus.cc:686] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 1 LEADER]: Becoming Leader. State: Replica: feca9b7cf6634a13aaa65a1b9c5e54f3, State: Running, Role: LEADER
I20260923 18:00:29.367169 21028 consensus_queue.cc:237] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff18e9c125e4f018e54c174e78ead29" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 40679 } } peers { permanent_uuid: "397270dade274d91949169fdb488c29b" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 44915 } } peers { permanent_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39369 } }
I20260923 18:00:29.370683 20553 catalog_manager.cc:5495] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to feca9b7cf6634a13aaa65a1b9c5e54f3 (127.15.139.130). New cstate: current_term: 1 leader_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff18e9c125e4f018e54c174e78ead29" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 40679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "397270dade274d91949169fdb488c29b" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 44915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "feca9b7cf6634a13aaa65a1b9c5e54f3" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39369 } health_report { overall_health: HEALTHY } } }
I20260923 18:00:29.411232 21021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.409770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48722 (local address 127.15.139.130:39369)
0923 18:00:29.409891 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:29.409895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:29.409908 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:00:29.410032 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:29.410035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:29.410104 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:29.410176 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:29.410908 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:29.410910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:29.410912 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:29.410922 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:29.410941 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:29.410949 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:29.411021 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:29.411095 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 18:00:29.411128 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 18:00:29.417567 20959 raft_consensus.cc:1261] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b [term 1 FOLLOWER]: Refusing update from remote peer feca9b7cf6634a13aaa65a1b9c5e54f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:29.417567 20697 raft_consensus.cc:1261] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29 [term 1 FOLLOWER]: Refusing update from remote peer feca9b7cf6634a13aaa65a1b9c5e54f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:29.418354 21032 consensus_queue.cc:1035] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "397270dade274d91949169fdb488c29b" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 44915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:29.418545 21028 consensus_queue.cc:1035] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fff18e9c125e4f018e54c174e78ead29" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 40679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:29.425747 21037 mvcc.cc:205] Tried to move back new op lower bound from 7332603614885498880 to 7332603614690746368. Current Snapshot: MvccSnapshot[applied={T|T < 7332603614885498880}]
I20260923 18:00:29.428098 21036 mvcc.cc:205] Tried to move back new op lower bound from 7332603614885498880 to 7332603614690746368. Current Snapshot: MvccSnapshot[applied={T|T < 7332603614885498880}]
I20260923 18:00:29.436326 20554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:29.436524 20554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:29.439056 20554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b110dc8b39b4028b3dfda2ae7784639: Sending DeleteTablet for 3 replicas of tablet efcd8ed626674dd8b56490dff64d9b92
I20260923 18:00:29.439888 20659 tablet_service.cc:1514] Processing DeleteTablet for tablet efcd8ed626674dd8b56490dff64d9b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:29 UTC) from {username='slave'} at 127.0.0.1:32866
I20260923 18:00:29.440145 20808 tablet_service.cc:1514] Processing DeleteTablet for tablet efcd8ed626674dd8b56490dff64d9b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:29 UTC) from {username='slave'} at 127.0.0.1:48714
I20260923 18:00:29.440449 21044 tablet_replica.cc:330] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29: stopping tablet replica
I20260923 18:00:29.440428 20939 tablet_service.cc:1514] Processing DeleteTablet for tablet efcd8ed626674dd8b56490dff64d9b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:29 UTC) from {username='slave'} at 127.0.0.1:57796
I20260923 18:00:29.440764 21044 raft_consensus.cc:2229] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:29.440886 21046 tablet_replica.cc:330] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b: stopping tablet replica
I20260923 18:00:29.441080 21044 raft_consensus.cc:2258] T efcd8ed626674dd8b56490dff64d9b92 P fff18e9c125e4f018e54c174e78ead29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:29.441187 21046 raft_consensus.cc:2229] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:29.441409 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 20615
I20260923 18:00:29.441527 21046 raft_consensus.cc:2258] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:29.443624 21046 ts_tablet_manager.cc:1905] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:29.444468 21045 tablet_replica.cc:330] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3: stopping tablet replica
I20260923 18:00:29.444747 21045 raft_consensus.cc:2229] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:29.445139 21045 raft_consensus.cc:2258] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:29.446491 21045 ts_tablet_manager.cc:1905] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:29.447333 21046 ts_tablet_manager.cc:1918] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:29.447516 21046 log.cc:1196] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/efcd8ed626674dd8b56490dff64d9b92
I20260923 18:00:29.447938 21046 ts_tablet_manager.cc:1939] T efcd8ed626674dd8b56490dff64d9b92 P 397270dade274d91949169fdb488c29b: Deleting consensus metadata
I20260923 18:00:29.449266 21045 ts_tablet_manager.cc:1918] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:29.449360 21045 log.cc:1196] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/efcd8ed626674dd8b56490dff64d9b92
I20260923 18:00:29.449311 20542 catalog_manager.cc:4841] TS 397270dade274d91949169fdb488c29b (127.15.139.131:44915): tablet efcd8ed626674dd8b56490dff64d9b92 (table test-table [id=c6d0f1e14b534ba9a280d86268969d81]) successfully deleted
I20260923 18:00:29.449676 21045 ts_tablet_manager.cc:1939] T efcd8ed626674dd8b56490dff64d9b92 P feca9b7cf6634a13aaa65a1b9c5e54f3: Deleting consensus metadata
W20260923 18:00:29.449836 20539 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:40679 (error 108)
I20260923 18:00:29.450037 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 20746
I20260923 18:00:29.451840 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.451571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:40679 (local address 127.0.0.1:32870)
0923 18:00:29.451706 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:29.451769 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:40679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260923 18:00:29.452180 20539 catalog_manager.cc:4584] TS fff18e9c125e4f018e54c174e78ead29 (127.15.139.129:40679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet efcd8ed626674dd8b56490dff64d9b92: Network error: Client connection negotiation failed: client connection to 127.15.139.129:40679: connect: Connection refused (error 111)
I20260923 18:00:29.455651 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 20877
I20260923 18:00:29.456744 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.455796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:39369 (local address 127.0.0.1:48728)
0923 18:00:29.456630 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:29.456679 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.130:39369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":756}
I20260923 18:00:29.461663 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 20524
2026-09-23T18:00:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1178 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260923 18:00:29.483259 15918 mini_oidc.cc:152] Starting JWKS server
I20260923 18:00:29.484935 15918 webserver.cc:460] Webserver started at https://127.0.0.1:35355/ using document root <none> and password file <none>
I20260923 18:00:29.485087 15918 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 18:00:29.485316 15918 webserver.cc:460] Webserver started at http://127.0.0.1:44853/ using document root <none> and password file <none>
2026-09-23T18:00:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:29Z Disabled control of system clock
I20260923 18:00:29.495133 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40725
--webserver_interface=127.15.139.190
--webserver_port=0
--builtin_ntp_servers=127.15.139.148:38069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:40725
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35355/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/testchainca.cert with env {}
W20260923 18:00:29.603907 21053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:29.604172 21053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:29.604211 21053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:29.608256 21053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:29.608311 21053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:29.608327 21053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:29.608345 21053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:29.608361 21053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 18:00:29.608400 21053 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 18:00:29.608430 21053 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35355/jwks.json
I20260923 18:00:29.613737 21053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:38069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:40725
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35355/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:29.614751 21053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:29.615784 21053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:29.624493 21058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:29.624502 21061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:29.624502 21059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:29.624815 21053 server_base.cc:1034] running on GCE node
I20260923 18:00:29.625447 21053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:29.626642 21053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:29.627872 21053 hybrid_clock.cc:648] HybridClock initialized: now 1790186429627853 us; error 29 us; skew 500 ppm
I20260923 18:00:29.630038 21053 webserver.cc:460] Webserver started at http://127.15.139.190:41227/ using document root <none> and password file <none>
I20260923 18:00:29.630695 21053 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:29.630800 21053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:29.631054 21053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:29.633364 21053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "668b69d9a87f4090bd5986e2071f150c"
format_stamp: "Formatted at 2026-09-23 18:00:29 on dist-test-slave-vr57"
server_key: "5a8e34489eb225000367a4fd439fda78"
server_key_iv: "a30fcf9feeb42ae135d9e609538aeb7f"
server_key_version: "encryptionkey@0"
I20260923 18:00:29.634001 21053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "668b69d9a87f4090bd5986e2071f150c"
format_stamp: "Formatted at 2026-09-23 18:00:29 on dist-test-slave-vr57"
server_key: "5a8e34489eb225000367a4fd439fda78"
server_key_iv: "a30fcf9feeb42ae135d9e609538aeb7f"
server_key_version: "encryptionkey@0"
I20260923 18:00:29.638057 21053 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260923 18:00:29.640695 21067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:29.642038 21053 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:29.642212 21053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "668b69d9a87f4090bd5986e2071f150c"
format_stamp: "Formatted at 2026-09-23 18:00:29 on dist-test-slave-vr57"
server_key: "5a8e34489eb225000367a4fd439fda78"
server_key_iv: "a30fcf9feeb42ae135d9e609538aeb7f"
server_key_version: "encryptionkey@0"
I20260923 18:00:29.642341 21053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:29.657552 21053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:29.658352 21053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:29.658565 21053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:29.666098 21053 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:40725
I20260923 18:00:29.666133 21119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:40725 every 8 connection(s)
I20260923 18:00:29.667104 21053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:29.670599 21120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:29.671079 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 21053
I20260923 18:00:29.671235 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:29.671589 15918 external_mini_cluster.cc:1428] Setting key 70a41e62b4980f2a294d8ed769b5f052
I20260923 18:00:29.677526 21120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c: Bootstrap starting.
I20260923 18:00:29.679984 21120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:29.680733 21120 log.cc:824] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:29.681119 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.672945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.15.139.190:40725)
0923 18:00:29.673593 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:29.673608 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:29.673642 (+ 34us) server_negotiation.cc:407] Connection header received
0923 18:00:29.674296 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:29.674313 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:29.674726 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:29.675077 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:29.675589 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:29.676548 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:29.677459 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:29.678005 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:29.678180 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:29.679187 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:29.679192 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:29.679205 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:29.679679 (+ 474us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:29.679741 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:29.679762 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:29.679958 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:29.680037 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 18:00:29.680321 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":136,"threads_started":1}
I20260923 18:00:29.682632 21120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c: No bootstrap required, opened a new log
I20260923 18:00:29.684065 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:00:29.686642 21120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668b69d9a87f4090bd5986e2071f150c" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40725 } }
I20260923 18:00:29.686851 21120 raft_consensus.cc:374] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:29.686884 21120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668b69d9a87f4090bd5986e2071f150c, State: Initialized, Role: FOLLOWER
I20260923 18:00:29.687312 21120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668b69d9a87f4090bd5986e2071f150c" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40725 } }
I20260923 18:00:29.687438 21120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:29.687480 21120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:29.687614 21120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:29.688467 21120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668b69d9a87f4090bd5986e2071f150c" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40725 } }
I20260923 18:00:29.688755 21120 leader_election.cc:302] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 668b69d9a87f4090bd5986e2071f150c; no voters:
I20260923 18:00:29.689016 21120 leader_election.cc:288] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:29.689169 21125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:29.689414 21125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [term 1 LEADER]: Becoming Leader. State: Replica: 668b69d9a87f4090bd5986e2071f150c, State: Running, Role: LEADER
I20260923 18:00:29.689760 21125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668b69d9a87f4090bd5986e2071f150c" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40725 } }
I20260923 18:00:29.690300 21120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:29.691556 21127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 668b69d9a87f4090bd5986e2071f150c. Latest consensus state: current_term: 1 leader_uuid: "668b69d9a87f4090bd5986e2071f150c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668b69d9a87f4090bd5986e2071f150c" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40725 } } }
I20260923 18:00:29.691692 21127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:29.691541 21126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "668b69d9a87f4090bd5986e2071f150c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668b69d9a87f4090bd5986e2071f150c" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40725 } } }
I20260923 18:00:29.691802 21126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:29.693321 21134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:29.695958 21134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:29.701537 21134 catalog_manager.cc:1347] Generated new cluster ID: 0c5bb01793054b97aed2fe58fb1a2892
I20260923 18:00:29.701619 21134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:29.712148 21134 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:29.713449 21134 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:29.728392 21134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 668b69d9a87f4090bd5986e2071f150c: Generated new TSK 0
I20260923 18:00:29.729280 21134 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 18:00:29.761844 21150 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260923 18:00:29.761991 21150 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 18:00:29.762506 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.740956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.15.139.190:40725)
0923 18:00:29.741098 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:29.741103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:29.741331 (+ 228us) server_negotiation.cc:407] Connection header received
0923 18:00:29.741539 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:29.741542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:29.741591 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:29.741680 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0923 18:00:29.741984 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:29.742552 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:29.743801 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:29.744021 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:29.744142 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:29.744526 (+ 384us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 18:00:29.762071 (+ 17545us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35355/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 18:00:29.762362 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44524: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35355/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260923 18:00:29.762578 21123 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44524: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35355/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260923 18:00:29.763747 21149 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:29.740872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40725 (local address 127.0.0.1:44524)
0923 18:00:29.741230 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:29.741246 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:29.741384 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:29.741719 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:29.741722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:29.741730 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0923 18:00:29.741853 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:29.741859 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:29.742661 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:29.742664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:29.743683 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:29.743693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:29.744175 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:29.744179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:29.744274 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:29.744331 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 18:00:29.762328 (+ 17997us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35355/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 18:00:29.762545 (+ 217us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.139.190:40725: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35355/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":120,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.139.190:40725: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35355/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260923 18:00:29.764480 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 21053
2026-09-23T18:00:29Z chronyd exiting
I20260923 18:00:29.802147 15918 test_util.cc:175] -----------------------------------------------
I20260923 18:00:29.802237 15918 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790186398863621-15918-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (320 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260923 18:00:29.803212 15918 mini_oidc.cc:152] Starting JWKS server
I20260923 18:00:29.804126 15918 webserver.cc:460] Webserver started at https://127.0.0.1:44669/ using document root <none> and password file <none>
I20260923 18:00:29.804207 15918 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 18:00:29.804412 15918 webserver.cc:460] Webserver started at http://127.0.0.1:33753/ using document root <none> and password file <none>
2026-09-23T18:00:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:29Z Disabled control of system clock
I20260923 18:00:29.813282 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:34121
--webserver_interface=127.15.139.190
--webserver_port=0
--builtin_ntp_servers=127.15.139.148:37985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:34121
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44669/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/testchainca.cert with env {}
W20260923 18:00:29.926452 21156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:29.926760 21156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:29.926807 21156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:29.931083 21156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:29.931159 21156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:29.931174 21156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:29.931192 21156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:29.931208 21156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 18:00:29.931226 21156 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 18:00:29.931239 21156 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44669/jwks.json
I20260923 18:00:29.936822 21156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:37985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:34121
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:34121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44669/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:29.937906 21156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:29.938992 21156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:29.947467 21161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:29.947530 21162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:29.947525 21164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:29.947753 21156 server_base.cc:1034] running on GCE node
I20260923 18:00:29.948693 21156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:29.950026 21156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:29.951242 21156 hybrid_clock.cc:648] HybridClock initialized: now 1790186429951224 us; error 38 us; skew 500 ppm
I20260923 18:00:29.953271 21156 webserver.cc:460] Webserver started at http://127.15.139.190:35229/ using document root <none> and password file <none>
I20260923 18:00:29.953902 21156 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:29.954005 21156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:29.954242 21156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:29.956432 21156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "100a66baa89a48c1a6aa250b184a7e7a"
format_stamp: "Formatted at 2026-09-23 18:00:29 on dist-test-slave-vr57"
server_key: "d141328de587fbac8b15106e93f54b45"
server_key_iv: "5881a88b7bda571e1560db5053eac9fa"
server_key_version: "encryptionkey@0"
I20260923 18:00:29.957194 21156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "100a66baa89a48c1a6aa250b184a7e7a"
format_stamp: "Formatted at 2026-09-23 18:00:29 on dist-test-slave-vr57"
server_key: "d141328de587fbac8b15106e93f54b45"
server_key_iv: "5881a88b7bda571e1560db5053eac9fa"
server_key_version: "encryptionkey@0"
I20260923 18:00:29.961221 21156 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:29.963800 21170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:29.965377 21156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:29.965564 21156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "100a66baa89a48c1a6aa250b184a7e7a"
format_stamp: "Formatted at 2026-09-23 18:00:29 on dist-test-slave-vr57"
server_key: "d141328de587fbac8b15106e93f54b45"
server_key_iv: "5881a88b7bda571e1560db5053eac9fa"
server_key_version: "encryptionkey@0"
I20260923 18:00:29.965711 21156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:29.978592 21156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:29.979406 21156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:29.979650 21156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:29.987216 21156 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:34121
I20260923 18:00:29.987237 21222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:34121 every 8 connection(s)
I20260923 18:00:29.988536 21156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:29.989524 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 21156
I20260923 18:00:29.989691 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:29.989998 15918 external_mini_cluster.cc:1428] Setting key fb6b18a7cfadd186a13f3a44b9df616f
I20260923 18:00:29.992682 21223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:29.998467 21223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a: Bootstrap starting.
I20260923 18:00:29.999712 21226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:29.991417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.15.139.190:34121)
0923 18:00:29.992183 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:29.992200 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:29.992237 (+ 37us) server_negotiation.cc:407] Connection header received
0923 18:00:29.992993 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:29.993013 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:29.993412 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:29.993785 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:29.994233 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:29.995229 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:29.996212 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:29.996685 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:29.996867 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:29.997818 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:29.997823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:29.997837 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:29.998315 (+ 478us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:29.998374 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:29.998395 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:29.998514 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:29.998667 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 18:00:29.998940 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":140,"threads_started":1}
I20260923 18:00:30.001009 21223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:30.001869 21223 log.cc:824] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:30.002707 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:00:30.004200 21223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a: No bootstrap required, opened a new log
I20260923 18:00:30.007984 21223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100a66baa89a48c1a6aa250b184a7e7a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34121 } }
I20260923 18:00:30.008287 21223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:30.008340 21223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 100a66baa89a48c1a6aa250b184a7e7a, State: Initialized, Role: FOLLOWER
I20260923 18:00:30.008783 21223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100a66baa89a48c1a6aa250b184a7e7a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34121 } }
I20260923 18:00:30.008908 21223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:30.008950 21223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:30.009023 21223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:30.009932 21223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100a66baa89a48c1a6aa250b184a7e7a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34121 } }
I20260923 18:00:30.010236 21223 leader_election.cc:302] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 100a66baa89a48c1a6aa250b184a7e7a; no voters:
I20260923 18:00:30.010505 21223 leader_election.cc:288] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:30.010721 21228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:30.010993 21228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [term 1 LEADER]: Becoming Leader. State: Replica: 100a66baa89a48c1a6aa250b184a7e7a, State: Running, Role: LEADER
I20260923 18:00:30.011308 21228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100a66baa89a48c1a6aa250b184a7e7a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34121 } }
I20260923 18:00:30.011897 21223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:30.013048 21230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 100a66baa89a48c1a6aa250b184a7e7a. Latest consensus state: current_term: 1 leader_uuid: "100a66baa89a48c1a6aa250b184a7e7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100a66baa89a48c1a6aa250b184a7e7a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34121 } } }
I20260923 18:00:30.013190 21230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:30.013471 21229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "100a66baa89a48c1a6aa250b184a7e7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100a66baa89a48c1a6aa250b184a7e7a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34121 } } }
I20260923 18:00:30.013562 21229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:30.013679 21237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:30.017095 21237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:30.023357 21237 catalog_manager.cc:1347] Generated new cluster ID: e939fde60e4143999ba2dfa5c3e8202a
I20260923 18:00:30.023447 21237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:30.044827 21237 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:30.045827 21237 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:30.057185 21237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 100a66baa89a48c1a6aa250b184a7e7a: Generated new TSK 0
I20260923 18:00:30.058040 21237 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 18:00:30.084566 21253 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260923 18:00:30.084683 21253 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 18:00:30.085372 21226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:30.065790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58648 (local address 127.15.139.190:34121)
0923 18:00:30.065966 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:30.065970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:30.066092 (+ 122us) server_negotiation.cc:407] Connection header received
0923 18:00:30.066225 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:30.066227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:30.066276 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:30.066375 (+ 99us) server_negotiation.cc:226] Negotiated authn=JWT
0923 18:00:30.066648 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.067235 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.068509 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.068718 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.068900 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:30.069214 (+ 314us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 18:00:30.084937 (+ 15723us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 18:00:30.085216 (+ 279us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":53}
W20260923 18:00:30.085460 21226 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260923 18:00:30.085573 21252 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:30.065649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:34121 (local address 127.0.0.1:58648)
0923 18:00:30.065965 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:30.065981 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:30.066096 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:30.066394 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:30.066397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:30.066412 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0923 18:00:30.066526 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:30.066532 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.067346 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.067350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:30.068366 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:30.068377 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.068915 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.068919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:30.069009 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:30.069037 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 18:00:30.085176 (+ 16139us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 18:00:30.085364 (+ 188us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.139.190:34121: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":102,"threads_started":1}
I20260923 18:00:30.086287 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 21156
2026-09-23T18:00:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (322 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260923 18:00:30.125849 15918 mini_oidc.cc:152] Starting JWKS server
I20260923 18:00:30.126754 15918 webserver.cc:460] Webserver started at https://127.0.0.1:33563/ using document root <none> and password file <none>
I20260923 18:00:30.126866 15918 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 18:00:30.127161 15918 webserver.cc:460] Webserver started at http://127.0.0.1:35669/ using document root <none> and password file <none>
2026-09-23T18:00:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:30Z Disabled control of system clock
I20260923 18:00:30.136384 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:44495
--webserver_interface=127.15.139.190
--webserver_port=0
--builtin_ntp_servers=127.15.139.148:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:44495
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33563/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 18:00:30.250048 21259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:30.250447 21259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:30.250543 21259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:30.254930 21259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:30.254992 21259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:30.255008 21259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:30.255025 21259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:30.255069 21259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 18:00:30.255105 21259 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 18:00:30.255124 21259 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33563/jwks.json
I20260923 18:00:30.260082 21259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:44495
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:44495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33563/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:30.261215 21259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:30.262684 21259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:30.271059 21265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:30.271044 21267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:30.271049 21264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:30.271278 21259 server_base.cc:1034] running on GCE node
I20260923 18:00:30.272003 21259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:30.273087 21259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:30.274260 21259 hybrid_clock.cc:648] HybridClock initialized: now 1790186430274230 us; error 40 us; skew 500 ppm
I20260923 18:00:30.276351 21259 webserver.cc:460] Webserver started at http://127.15.139.190:36203/ using document root <none> and password file <none>
I20260923 18:00:30.276944 21259 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:30.277014 21259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:30.277261 21259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:30.279354 21259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "6b9fe5e27a3d43a5b9bc38d9f7f5ba2b"
format_stamp: "Formatted at 2026-09-23 18:00:30 on dist-test-slave-vr57"
server_key: "6538cd3b82a83124845d967c7b9bab61"
server_key_iv: "af0c03322ea9ebe870f6b22db30f16df"
server_key_version: "encryptionkey@0"
I20260923 18:00:30.280009 21259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "6b9fe5e27a3d43a5b9bc38d9f7f5ba2b"
format_stamp: "Formatted at 2026-09-23 18:00:30 on dist-test-slave-vr57"
server_key: "6538cd3b82a83124845d967c7b9bab61"
server_key_iv: "af0c03322ea9ebe870f6b22db30f16df"
server_key_version: "encryptionkey@0"
I20260923 18:00:30.283759 21259 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:00:30.286355 21273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:30.287731 21259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:30.287865 21259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "6b9fe5e27a3d43a5b9bc38d9f7f5ba2b"
format_stamp: "Formatted at 2026-09-23 18:00:30 on dist-test-slave-vr57"
server_key: "6538cd3b82a83124845d967c7b9bab61"
server_key_iv: "af0c03322ea9ebe870f6b22db30f16df"
server_key_version: "encryptionkey@0"
I20260923 18:00:30.287987 21259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:30.307921 21259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:30.308653 21259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:30.308864 21259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:30.315985 21259 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:44495
I20260923 18:00:30.316011 21325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:44495 every 8 connection(s)
I20260923 18:00:30.316996 21259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:30.320251 21326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:30.322782 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 21259
I20260923 18:00:30.322916 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:30.323271 15918 external_mini_cluster.cc:1428] Setting key 4f12e711a8821b0eae77bc5651b1814b
I20260923 18:00:30.326494 21326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b: Bootstrap starting.
I20260923 18:00:30.329156 21326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:30.329998 21326 log.cc:824] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:30.332273 21326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b: No bootstrap required, opened a new log
I20260923 18:00:30.333318 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:30.324801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41748 (local address 127.15.139.190:44495)
0923 18:00:30.325501 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:30.325518 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:30.325553 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:00:30.326162 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:30.326180 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:30.326637 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:30.326938 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:30.327378 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.328502 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.329718 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.330274 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.330444 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:30.331269 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:30.331277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:30.331292 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:30.331817 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:30.331886 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:30.331912 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:30.332018 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:30.332169 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 18:00:30.332428 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":206,"threads_started":1}
I20260923 18:00:30.336272 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:00:30.336508 21326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9fe5e27a3d43a5b9bc38d9f7f5ba2b" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44495 } }
I20260923 18:00:30.336740 21326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:30.336810 21326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b, State: Initialized, Role: FOLLOWER
I20260923 18:00:30.337288 21326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9fe5e27a3d43a5b9bc38d9f7f5ba2b" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44495 } }
I20260923 18:00:30.337448 21326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:30.337514 21326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:30.337617 21326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:30.338605 21326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9fe5e27a3d43a5b9bc38d9f7f5ba2b" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44495 } }
I20260923 18:00:30.338945 21326 leader_election.cc:302] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b; no voters:
I20260923 18:00:30.339253 21326 leader_election.cc:288] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:30.339404 21331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:30.339679 21331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [term 1 LEADER]: Becoming Leader. State: Replica: 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b, State: Running, Role: LEADER
I20260923 18:00:30.340031 21331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9fe5e27a3d43a5b9bc38d9f7f5ba2b" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44495 } }
I20260923 18:00:30.340484 21326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:30.342052 21332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b9fe5e27a3d43a5b9bc38d9f7f5ba2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9fe5e27a3d43a5b9bc38d9f7f5ba2b" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44495 } } }
I20260923 18:00:30.342103 21333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b. Latest consensus state: current_term: 1 leader_uuid: "6b9fe5e27a3d43a5b9bc38d9f7f5ba2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9fe5e27a3d43a5b9bc38d9f7f5ba2b" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44495 } } }
I20260923 18:00:30.342195 21332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:30.342202 21333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:30.342564 21343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:30.345566 21343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:30.351285 21343 catalog_manager.cc:1347] Generated new cluster ID: 391aacfde5f44394998d0fdc2f7c4b43
I20260923 18:00:30.351395 21343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:30.360100 21343 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:30.361100 21343 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:30.369848 21343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b9fe5e27a3d43a5b9bc38d9f7f5ba2b: Generated new TSK 0
I20260923 18:00:30.370671 21343 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:00:30.389537 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:30.384081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41758 (local address 127.15.139.190:44495)
0923 18:00:30.384233 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:30.384238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:30.384458 (+ 220us) server_negotiation.cc:407] Connection header received
0923 18:00:30.384666 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:30.384669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:30.384744 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:30.384835 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0923 18:00:30.385143 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.385739 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.387054 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.387287 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.387389 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:30.387769 (+ 380us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 18:00:30.389134 (+ 1365us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33563/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 18:00:30.389388 (+ 254us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41758: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33563/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":61}
W20260923 18:00:30.389613 21329 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41758: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33563/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260923 18:00:30.389624 21355 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:30.384025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44495 (local address 127.0.0.1:41758)
0923 18:00:30.384355 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:30.384372 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:30.384511 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:30.384867 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:30.384871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:30.384879 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0923 18:00:30.384999 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:30.385005 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.385865 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.385869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:30.386922 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:30.386933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.387414 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.387418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:30.387528 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:30.387566 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 18:00:30.389300 (+ 1734us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33563/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 18:00:30.389453 (+ 153us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.139.190:44495: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33563/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":126,"threads_started":1}
I20260923 18:00:30.390583 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 21259
E20260923 18:00:30.402572 21356 webserver.cc:533] Webserver: sslize failed:
E20260923 18:00:30.402647 21356 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-23T18:00:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (304 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-23T18:00:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:30Z Disabled control of system clock
I20260923 18:00:30.439456 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:37767
--webserver_interface=127.15.139.190
--webserver_port=0
--builtin_ntp_servers=127.15.139.148:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:37767
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/testchainca.cert with env {}
W20260923 18:00:30.552933 21360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:30.553220 21360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:30.553264 21360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:30.557376 21360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:30.557468 21360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:30.557485 21360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:30.557502 21360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:30.557523 21360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 18:00:30.557546 21360 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 18:00:30.557567 21360 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260923 18:00:30.562850 21360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:37767
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:37767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:30.563859 21360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:30.564862 21360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:30.572780 21366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:30.572788 21368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:30.572845 21365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:30.573026 21360 server_base.cc:1034] running on GCE node
I20260923 18:00:30.573637 21360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:30.574764 21360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:30.575953 21360 hybrid_clock.cc:648] HybridClock initialized: now 1790186430575942 us; error 30 us; skew 500 ppm
I20260923 18:00:30.577883 21360 webserver.cc:460] Webserver started at http://127.15.139.190:42067/ using document root <none> and password file <none>
I20260923 18:00:30.578462 21360 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:30.578554 21360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:30.578773 21360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:30.580850 21360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "56578e762117420891a5cd4c032b335f"
format_stamp: "Formatted at 2026-09-23 18:00:30 on dist-test-slave-vr57"
server_key: "eb09384227f507b1b58f39719425febb"
server_key_iv: "a358528c4a85d9c4d8cabb5fa4e8e3b2"
server_key_version: "encryptionkey@0"
I20260923 18:00:30.581418 21360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "56578e762117420891a5cd4c032b335f"
format_stamp: "Formatted at 2026-09-23 18:00:30 on dist-test-slave-vr57"
server_key: "eb09384227f507b1b58f39719425febb"
server_key_iv: "a358528c4a85d9c4d8cabb5fa4e8e3b2"
server_key_version: "encryptionkey@0"
I20260923 18:00:30.585187 21360 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260923 18:00:30.587690 21374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:30.588913 21360 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:00:30.589072 21360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "56578e762117420891a5cd4c032b335f"
format_stamp: "Formatted at 2026-09-23 18:00:30 on dist-test-slave-vr57"
server_key: "eb09384227f507b1b58f39719425febb"
server_key_iv: "a358528c4a85d9c4d8cabb5fa4e8e3b2"
server_key_version: "encryptionkey@0"
I20260923 18:00:30.589190 21360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:30.607640 21360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:30.608368 21360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:30.608570 21360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:30.615327 21360 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:37767
I20260923 18:00:30.615339 21426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:37767 every 8 connection(s)
I20260923 18:00:30.616336 21360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:30.619201 21427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:30.624543 21427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f: Bootstrap starting.
I20260923 18:00:30.625062 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 21360
I20260923 18:00:30.625228 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:30.625591 15918 external_mini_cluster.cc:1428] Setting key c12312680ddf2d9b9fa5135bbe0fd491
I20260923 18:00:30.627465 21427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:30.628263 21427 log.cc:824] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:30.630160 21427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f: No bootstrap required, opened a new log
I20260923 18:00:30.633540 21427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56578e762117420891a5cd4c032b335f" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37767 } }
I20260923 18:00:30.633776 21427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:30.633832 21427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56578e762117420891a5cd4c032b335f, State: Initialized, Role: FOLLOWER
I20260923 18:00:30.634316 21427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56578e762117420891a5cd4c032b335f" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37767 } }
I20260923 18:00:30.634470 21427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:30.634560 21427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:30.634684 21427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:30.634948 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:30.626896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38206 (local address 127.15.139.190:37767)
0923 18:00:30.627613 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:30.627631 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:30.627667 (+ 36us) server_negotiation.cc:407] Connection header received
0923 18:00:30.628289 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:30.628306 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:30.628748 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:30.629088 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:30.629499 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.630509 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.631451 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.632069 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.632244 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:30.633099 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:30.633104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:30.633117 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:00:30.633483 (+ 366us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:30.633535 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:00:30.633557 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:30.633661 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:30.633797 (+ 136us) server_negotiation.cc:299] Negotiation successful
0923 18:00:30.634082 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":128,"threads_started":1}
I20260923 18:00:30.635823 21427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56578e762117420891a5cd4c032b335f" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37767 } }
I20260923 18:00:30.636175 21427 leader_election.cc:302] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56578e762117420891a5cd4c032b335f; no voters:
I20260923 18:00:30.636480 21427 leader_election.cc:288] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:30.636607 21432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:30.636807 21432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [term 1 LEADER]: Becoming Leader. State: Replica: 56578e762117420891a5cd4c032b335f, State: Running, Role: LEADER
I20260923 18:00:30.637095 21432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56578e762117420891a5cd4c032b335f" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37767 } }
I20260923 18:00:30.637806 21427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:30.638155 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:00:30.638728 21433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56578e762117420891a5cd4c032b335f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56578e762117420891a5cd4c032b335f" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37767 } } }
I20260923 18:00:30.638839 21433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:30.639051 21433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56578e762117420891a5cd4c032b335f. Latest consensus state: current_term: 1 leader_uuid: "56578e762117420891a5cd4c032b335f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56578e762117420891a5cd4c032b335f" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37767 } } }
I20260923 18:00:30.639128 21433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:30.639415 21441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:30.642407 21441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:30.648232 21441 catalog_manager.cc:1347] Generated new cluster ID: c331f6201f3e478fbf4fa63b1a8d7a37
I20260923 18:00:30.648304 21441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:30.666466 21441 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:30.667418 21441 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:30.674234 21441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56578e762117420891a5cd4c032b335f: Generated new TSK 0
I20260923 18:00:30.674955 21441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:00:30.777172 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:30.688579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38212 (local address 127.15.139.190:37767)
0923 18:00:30.688713 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:30.688717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:30.688876 (+ 159us) server_negotiation.cc:407] Connection header received
0923 18:00:30.689031 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:30.689034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:30.689083 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:30.689180 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0923 18:00:30.689447 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.689993 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.691259 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.691603 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.691712 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:30.692083 (+ 371us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 18:00:30.776784 (+ 84701us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 18:00:30.777055 (+ 271us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38212: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":51}
W20260923 18:00:30.777269 21430 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38212: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260923 18:00:30.777505 21456 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:30.688433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:37767 (local address 127.0.0.1:38212)
0923 18:00:30.688778 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:30.688793 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:30.688910 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:30.689210 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:30.689213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:30.689222 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0923 18:00:30.689344 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:30.689350 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.690108 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.690112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:30.691131 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:30.691141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:30.691778 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:30.691783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:30.691883 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:30.691914 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 18:00:30.777045 (+ 85131us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 18:00:30.777281 (+ 236us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.139.190:37767: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":111,"threads_started":1}
I20260923 18:00:30.778239 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 21360
2026-09-23T18:00:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (367 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (125 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (132 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:31 dist-test-slave-vr57 krb5kdc[21468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:31 dist-test-slave-vr57 krb5kdc[21468](info): set up 2 sockets
Sep 23 18:00:31 dist-test-slave-vr57 krb5kdc[21468](info): commencing operation
krb5kdc: starting...
W20260923 18:00:32.140486 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:32 dist-test-slave-vr57 krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:00:32.266403 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:35143
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:35143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:32.380728 21481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:32.381031 21481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:32.381112 21481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:32.385181 21481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:32.385258 21481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:32.385280 21481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:32.385298 21481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:32.385313 21481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:32.391036 21481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:35143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:35143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:32.392237 21481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:32.393296 21481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:32.402446 21489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:32.402449 21487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:32.402693 21481 server_base.cc:1034] running on GCE node
W20260923 18:00:32.402516 21486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:32.403437 21481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:32.404696 21481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:32.405902 21481 hybrid_clock.cc:648] HybridClock initialized: now 1790186432405884 us; error 33 us; skew 500 ppm
Sep 23 18:00:32 dist-test-slave-vr57 krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186432, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:32.414402 21481 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:00:32.415556 21481 webserver.cc:460] Webserver started at http://127.15.139.190:39721/ using document root <none> and password file <none>
I20260923 18:00:32.416196 21481 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:32.416265 21481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:32.416440 21481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:32.418618 21481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "b08eb1297ae14f0fb24f4420d201cee2"
format_stamp: "Formatted at 2026-09-23 18:00:32 on dist-test-slave-vr57"
server_key: "dbc1e49f921021d7dd3eebca34b27492"
server_key_iv: "631bfdffd1630924d298876c5c725194"
server_key_version: "encryptionkey@0"
I20260923 18:00:32.419175 21481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "b08eb1297ae14f0fb24f4420d201cee2"
format_stamp: "Formatted at 2026-09-23 18:00:32 on dist-test-slave-vr57"
server_key: "dbc1e49f921021d7dd3eebca34b27492"
server_key_iv: "631bfdffd1630924d298876c5c725194"
server_key_version: "encryptionkey@0"
I20260923 18:00:32.423332 21481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260923 18:00:32.425935 21496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:32.427313 21481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:32.427477 21481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "b08eb1297ae14f0fb24f4420d201cee2"
format_stamp: "Formatted at 2026-09-23 18:00:32 on dist-test-slave-vr57"
server_key: "dbc1e49f921021d7dd3eebca34b27492"
server_key_iv: "631bfdffd1630924d298876c5c725194"
server_key_version: "encryptionkey@0"
I20260923 18:00:32.427649 21481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:32.439613 21481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:32.442781 21481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:32.442994 21481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:32.450063 21481 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:35143
I20260923 18:00:32.450103 21548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:35143 every 8 connection(s)
I20260923 18:00:32.451200 21481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:32.452965 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 21481
I20260923 18:00:32.453117 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:32.453410 15918 external_mini_cluster.cc:1428] Setting key f1ebceb5b83a0bfdf714c1e01e985eb8
I20260923 18:00:32.454478 21549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:32.462083 21549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2: Bootstrap starting.
Sep 23 18:00:32 dist-test-slave-vr57 krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:32.464774 21549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:32.465898 21549 log.cc:824] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:32.468482 21549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2: No bootstrap required, opened a new log
I20260923 18:00:32.469934 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:32.454927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58016 (local address 127.15.139.190:35143)
0923 18:00:32.455452 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:32.455476 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:32.455562 (+ 86us) server_negotiation.cc:407] Connection header received
0923 18:00:32.456506 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:32.456531 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:32.456975 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:32.457404 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:32.458190 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:32.459857 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:32.460775 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:32.461531 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:32.461724 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:32.464423 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:32.464430 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:32.464448 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:32.464484 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:32.466643 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:32.467101 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:32.467108 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:32.467114 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:32.467260 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:32.467569 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:32.467574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:32.467578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:32.468150 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:32.468358 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:32.468582 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 18:00:32.468919 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":139,"threads_started":1}
I20260923 18:00:32.472757 21549 raft_consensus.cc:348] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b08eb1297ae14f0fb24f4420d201cee2" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35143 } }
I20260923 18:00:32.473050 21549 raft_consensus.cc:374] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:32.473094 21549 raft_consensus.cc:729] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b08eb1297ae14f0fb24f4420d201cee2, State: Initialized, Role: FOLLOWER
I20260923 18:00:32.473479 21549 consensus_queue.cc:260] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b08eb1297ae14f0fb24f4420d201cee2" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35143 } }
I20260923 18:00:32.473613 21549 raft_consensus.cc:388] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:32.473654 21549 raft_consensus.cc:482] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:32.473731 21549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:32.474660 21549 raft_consensus.cc:504] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b08eb1297ae14f0fb24f4420d201cee2" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35143 } }
I20260923 18:00:32.475039 21549 leader_election.cc:302] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b08eb1297ae14f0fb24f4420d201cee2; no voters:
I20260923 18:00:32.475363 21549 leader_election.cc:288] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:32.475580 21554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:32.475855 21554 raft_consensus.cc:686] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [term 1 LEADER]: Becoming Leader. State: Replica: b08eb1297ae14f0fb24f4420d201cee2, State: Running, Role: LEADER
I20260923 18:00:32.476178 21554 consensus_queue.cc:237] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b08eb1297ae14f0fb24f4420d201cee2" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35143 } }
I20260923 18:00:32.476717 21549 sys_catalog.cc:564] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:32.478029 21555 sys_catalog.cc:455] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b08eb1297ae14f0fb24f4420d201cee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b08eb1297ae14f0fb24f4420d201cee2" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35143 } } }
I20260923 18:00:32.478197 21555 sys_catalog.cc:458] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:32.478581 21556 sys_catalog.cc:455] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b08eb1297ae14f0fb24f4420d201cee2. Latest consensus state: current_term: 1 leader_uuid: "b08eb1297ae14f0fb24f4420d201cee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b08eb1297ae14f0fb24f4420d201cee2" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35143 } } }
I20260923 18:00:32.478735 21556 sys_catalog.cc:458] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2 [sys.catalog]: This master's current role is: LEADER
W20260923 18:00:32.482061 21569 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:00:32.482148 21569 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:00:32.482218 21563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:32.483011 21563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:32.488961 21563 catalog_manager.cc:1347] Generated new cluster ID: a268e01564b1465eb93710042f3fb14e
I20260923 18:00:32.489048 21563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:32.500144 21563 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:32.501119 21563 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:32.512195 21563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b08eb1297ae14f0fb24f4420d201cee2: Generated new TSK 0
I20260923 18:00:32.512948 21563 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:00:32.544611 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35143
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:32.660655 21575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:32.660938 21575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:32.660987 21575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:32.665091 21575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:32.665155 21575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:32.665213 21575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:32.670411 21575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:32.671527 21575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:32.672488 21575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:32.682410 21581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:32.682619 21575 server_base.cc:1034] running on GCE node
W20260923 18:00:32.682821 21580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:32.682838 21583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:32.683357 21575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:32.684052 21575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:32.685222 21575 hybrid_clock.cc:648] HybridClock initialized: now 1790186432685198 us; error 38 us; skew 500 ppm
Sep 23 18:00:32 dist-test-slave-vr57 krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186432, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:32.688130 21575 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:00:32.689016 21575 webserver.cc:460] Webserver started at http://127.15.139.129:40895/ using document root <none> and password file <none>
I20260923 18:00:32.689597 21575 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:32.689661 21575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:32.689891 21575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:32.692165 21575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "f73864212dc5417188b89964ae4ba299"
format_stamp: "Formatted at 2026-09-23 18:00:32 on dist-test-slave-vr57"
server_key: "e956af63eb37fbd38f58978c1750e15f"
server_key_iv: "c6618645838c186ef9f18a71c0fcd426"
server_key_version: "encryptionkey@0"
I20260923 18:00:32.692759 21575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "f73864212dc5417188b89964ae4ba299"
format_stamp: "Formatted at 2026-09-23 18:00:32 on dist-test-slave-vr57"
server_key: "e956af63eb37fbd38f58978c1750e15f"
server_key_iv: "c6618645838c186ef9f18a71c0fcd426"
server_key_version: "encryptionkey@0"
I20260923 18:00:32.696882 21575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:00:32.699390 21590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:32.700784 21575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:00:32.700937 21575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "f73864212dc5417188b89964ae4ba299"
format_stamp: "Formatted at 2026-09-23 18:00:32 on dist-test-slave-vr57"
server_key: "e956af63eb37fbd38f58978c1750e15f"
server_key_iv: "c6618645838c186ef9f18a71c0fcd426"
server_key_version: "encryptionkey@0"
I20260923 18:00:32.701069 21575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:32.728478 21575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:32.731935 21575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:32.732151 21575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:32.732785 21575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:32.733566 21575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:32.733644 21575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:32.733711 21575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:32.733752 21575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:32.743104 21575 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:41657
I20260923 18:00:32.743165 21703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:41657 every 8 connection(s)
I20260923 18:00:32.744128 21575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:32.751708 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 21575
I20260923 18:00:32.751840 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:32.752172 15918 external_mini_cluster.cc:1428] Setting key c37c8549c11dd1f9a572bda63d7acb75
Sep 23 18:00:32 dist-test-slave-vr57 krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186432, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:32.759384 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:32.746237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:48713 (local address 127.15.139.190:35143)
0923 18:00:32.746380 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:32.746384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:32.747249 (+ 865us) server_negotiation.cc:407] Connection header received
0923 18:00:32.748775 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:32.748779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:32.748839 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:32.748950 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:32.750152 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:32.750699 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:32.752119 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:32.752414 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:32.752503 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:32.755626 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:32.755630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:32.755633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:32.755667 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:32.757124 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:32.757640 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:32.757643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:32.757645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:32.757757 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:32.758224 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:32.758227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:32.758228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:32.758397 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:32.758482 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:32.759155 (+ 673us) server_negotiation.cc:299] Negotiation successful
0923 18:00:32.759264 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 18:00:32.760691 21706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:32.746644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:35143 (local address 127.15.139.129:48713)
0923 18:00:32.747101 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:32.747143 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:32.748558 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:32.749138 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:32.749145 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:32.749517 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:32.750005 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:32.750018 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:32.750828 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:32.750836 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:32.751994 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:32.752005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:32.752594 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:32.752598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:32.752773 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:32.753512 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:32.753534 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:32.755421 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:32.757251 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:32.757257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:32.757272 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:32.757532 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:32.757885 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:32.757892 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:32.757895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:32.758117 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:32.758504 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:32.758514 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:32.758872 (+ 358us) client_negotiation.cc:769] Sending connection context
0923 18:00:32.759105 (+ 233us) client_negotiation.cc:240] Negotiation successful
0923 18:00:32.759536 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260923 18:00:32.762060 21704 heartbeater.cc:347] Connected to a master server at 127.15.139.190:35143
I20260923 18:00:32.762315 21704 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
I20260923 18:00:32.762876 21704 heartbeater.cc:511] Master 127.15.139.190:35143 requested a full tablet report, sending...
I20260923 18:00:32.764619 21513 ts_manager.cc:194] Registered new tserver with Master: f73864212dc5417188b89964ae4ba299 (127.15.139.129:41657)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:00:32.765843 21513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:48713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:00:32.780864 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35143
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:32.899865 21709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:32.900152 21709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:32.900225 21709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:32.904429 21709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:32.904497 21709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:32.904601 21709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:32.910378 21709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:32.911485 21709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:32.912549 21709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:32.922348 21715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:32.922410 21714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:32.922506 21717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:32.922415 21709 server_base.cc:1034] running on GCE node
I20260923 18:00:32.922940 21709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:32.923614 21709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:32.924803 21709 hybrid_clock.cc:648] HybridClock initialized: now 1790186432924765 us; error 46 us; skew 500 ppm
Sep 23 18:00:32 dist-test-slave-vr57 krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186432, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:32.927949 21709 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:00:32.929028 21709 webserver.cc:460] Webserver started at http://127.15.139.130:39683/ using document root <none> and password file <none>
I20260923 18:00:32.929679 21709 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:32.929777 21709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:32.930048 21709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:32.932610 21709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "d926afc54ff144ee9119f96dfa5e422b"
format_stamp: "Formatted at 2026-09-23 18:00:32 on dist-test-slave-vr57"
server_key: "b4ebfa47e57604449383d5132d9c9dc6"
server_key_iv: "2d796f2c56f43ae434c914469e9bbe2c"
server_key_version: "encryptionkey@0"
I20260923 18:00:32.933255 21709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "d926afc54ff144ee9119f96dfa5e422b"
format_stamp: "Formatted at 2026-09-23 18:00:32 on dist-test-slave-vr57"
server_key: "b4ebfa47e57604449383d5132d9c9dc6"
server_key_iv: "2d796f2c56f43ae434c914469e9bbe2c"
server_key_version: "encryptionkey@0"
I20260923 18:00:32.937772 21709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:32.940670 21724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:32.941972 21709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:32.942114 21709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "d926afc54ff144ee9119f96dfa5e422b"
format_stamp: "Formatted at 2026-09-23 18:00:32 on dist-test-slave-vr57"
server_key: "b4ebfa47e57604449383d5132d9c9dc6"
server_key_iv: "2d796f2c56f43ae434c914469e9bbe2c"
server_key_version: "encryptionkey@0"
I20260923 18:00:32.942230 21709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:32.952992 21709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:32.956133 21709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:32.956358 21709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:32.957006 21709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:32.957875 21709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:32.957952 21709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:32.958021 21709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:32.958065 21709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:32.967072 21709 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:37811
I20260923 18:00:32.967135 21837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:37811 every 8 connection(s)
I20260923 18:00:32.968119 21709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:32.978027 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 21709
I20260923 18:00:32.978183 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:32.978551 15918 external_mini_cluster.cc:1428] Setting key 9ec1d06dcf5c2e6eb9a9ff3907b6b7ec
Sep 23 18:00:32 dist-test-slave-vr57 krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186432, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:32.986156 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:32.970076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:41625 (local address 127.15.139.190:35143)
0923 18:00:32.970233 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:32.970237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:32.971151 (+ 914us) server_negotiation.cc:407] Connection header received
0923 18:00:32.972879 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:32.972884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:32.972956 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:32.973053 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:32.974370 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:32.974991 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:32.976440 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:32.976733 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:32.976859 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:32.980942 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:32.980949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:32.980954 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:32.980986 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:32.983240 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:32.984027 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:32.984031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:32.984033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:32.984152 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:32.984603 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:32.984606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:32.984608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:32.984873 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:32.984968 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:32.985862 (+ 894us) server_negotiation.cc:299] Negotiation successful
0923 18:00:32.986032 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 18:00:32.987362 21840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:32.970454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:35143 (local address 127.15.139.130:41625)
0923 18:00:32.970964 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:32.971016 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:32.972618 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:32.973240 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:32.973248 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:32.973663 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:32.974198 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:32.974213 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:32.975149 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:32.975156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:32.976251 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:32.976262 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:32.976900 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:32.976907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:32.977155 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:32.978383 (+ 1228us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:32.978419 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:32.980713 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:32.983404 (+ 2691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:32.983409 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:32.983423 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:32.983847 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:32.984271 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:32.984276 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:32.984280 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:32.984490 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:32.985172 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:32.985182 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:32.985532 (+ 350us) client_negotiation.cc:769] Sending connection context
0923 18:00:32.985754 (+ 222us) client_negotiation.cc:240] Negotiation successful
0923 18:00:32.986152 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":111,"threads_started":1}
I20260923 18:00:32.988811 21838 heartbeater.cc:347] Connected to a master server at 127.15.139.190:35143
I20260923 18:00:32.989142 21838 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:32.989666 21838 heartbeater.cc:511] Master 127.15.139.190:35143 requested a full tablet report, sending...
I20260923 18:00:32.990906 21513 ts_manager.cc:194] Registered new tserver with Master: d926afc54ff144ee9119f96dfa5e422b (127.15.139.130:37811)
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
I20260923 18:00:32.991730 21513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:41625
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:00:33.011085 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35143
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:33.122766 21843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:33.123045 21843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:33.123090 21843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:33.127264 21843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:33.127334 21843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:33.127396 21843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:33.133158 21843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:33.134454 21843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:33.135563 21843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:33.145514 21849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:33.145577 21851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:33.145537 21848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:33.145764 21843 server_base.cc:1034] running on GCE node
I20260923 18:00:33.146345 21843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:33.147075 21843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:33.148232 21843 hybrid_clock.cc:648] HybridClock initialized: now 1790186433148212 us; error 35 us; skew 500 ppm
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186433, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:33.151046 21843 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:00:33.151925 21843 webserver.cc:460] Webserver started at http://127.15.139.131:42181/ using document root <none> and password file <none>
I20260923 18:00:33.152478 21843 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:33.152585 21843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:33.152819 21843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:33.154800 21843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "d424fb430cea460ca97332bce80f732c"
format_stamp: "Formatted at 2026-09-23 18:00:33 on dist-test-slave-vr57"
server_key: "01726ca1597749f6a59b2a9c03795fa0"
server_key_iv: "50588e8b1bbe2310120098475755b520"
server_key_version: "encryptionkey@0"
I20260923 18:00:33.155351 21843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "d424fb430cea460ca97332bce80f732c"
format_stamp: "Formatted at 2026-09-23 18:00:33 on dist-test-slave-vr57"
server_key: "01726ca1597749f6a59b2a9c03795fa0"
server_key_iv: "50588e8b1bbe2310120098475755b520"
server_key_version: "encryptionkey@0"
I20260923 18:00:33.159099 21843 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:33.161522 21858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:33.162640 21843 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:33.162766 21843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "d424fb430cea460ca97332bce80f732c"
format_stamp: "Formatted at 2026-09-23 18:00:33 on dist-test-slave-vr57"
server_key: "01726ca1597749f6a59b2a9c03795fa0"
server_key_iv: "50588e8b1bbe2310120098475755b520"
server_key_version: "encryptionkey@0"
I20260923 18:00:33.162869 21843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:33.183632 21843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:33.187063 21843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:33.187263 21843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:33.187916 21843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:33.188789 21843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:33.188859 21843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:33.188934 21843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:33.188983 21843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:33.197849 21843 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:37539
I20260923 18:00:33.197901 21971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:37539 every 8 connection(s)
I20260923 18:00:33.198772 21843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:33.207989 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 21843
I20260923 18:00:33.208127 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186433, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:33.208456 15918 external_mini_cluster.cc:1428] Setting key 2b58468b735d63dc8fb100b62953758a
I20260923 18:00:33.212741 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:33.200675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:50943 (local address 127.15.139.190:35143)
0923 18:00:33.200819 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:33.200823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:33.201649 (+ 826us) server_negotiation.cc:407] Connection header received
0923 18:00:33.202923 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:33.202926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:33.202973 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:33.203048 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:33.204264 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:33.204794 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:33.206023 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:33.206231 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:33.206311 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:33.208885 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:33.208907 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:33.208913 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:33.208955 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:33.210693 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:33.211175 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:33.211179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:33.211181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:33.211275 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:33.211669 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:33.211673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:33.211675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:33.211862 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:33.211990 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:33.212475 (+ 485us) server_negotiation.cc:299] Negotiation successful
0923 18:00:33.212592 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 18:00:33.213588 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:33.201037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:35143 (local address 127.15.139.131:50943)
0923 18:00:33.201480 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:33.201539 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:33.202736 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:33.203232 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:33.203239 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:33.203656 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:33.204130 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:33.204144 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:33.204897 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:33.204899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:33.205863 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:33.205880 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:33.206355 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:33.206357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:33.206495 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:33.207015 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:33.207030 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:33.208702 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:33.210795 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:33.210801 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:33.210815 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:33.211071 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:33.211369 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:33.211374 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:33.211376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:33.211567 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:33.211990 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:33.211998 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:33.212239 (+ 241us) client_negotiation.cc:769] Sending connection context
0923 18:00:33.212414 (+ 175us) client_negotiation.cc:240] Negotiation successful
0923 18:00:33.212746 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":115,"threads_started":1}
I20260923 18:00:33.214628 21972 heartbeater.cc:347] Connected to a master server at 127.15.139.190:35143
I20260923 18:00:33.214865 21972 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:33.215366 21972 heartbeater.cc:511] Master 127.15.139.190:35143 requested a full tablet report, sending...
I20260923 18:00:33.216372 21513 ts_manager.cc:194] Registered new tserver with Master: d424fb430cea460ca97332bce80f732c (127.15.139.131:37539)
I20260923 18:00:33.216923 21513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:50943
I20260923 18:00:33.222326 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/test-admin.keytab.
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:33.239984 15918 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260923 18:00:33.243000 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:33.241328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.15.139.129:41657)
0923 18:00:33.241670 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:33.241689 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:33.241719 (+ 30us) server_negotiation.cc:407] Connection header received
0923 18:00:33.241782 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:33.241787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:33.242047 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:33.242209 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:33.242793 (+ 584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37216: BlockingRecv error: recv got EOF from 127.0.0.1:37216 (error 108)
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":120,"threads_started":1}
W20260923 18:00:33.243086 15918 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/krb5cc' of type FILE
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:33.244241 15918 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:33.245304 15918 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.129@KRBTEST.COM
I20260923 18:00:33.255084 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:33.246251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37232 (local address 127.15.139.129:41657)
0923 18:00:33.246386 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:33.246389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:33.246500 (+ 111us) server_negotiation.cc:407] Connection header received
0923 18:00:33.246686 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:33.246689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:33.246739 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:33.246807 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:33.247378 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:33.248272 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:33.249083 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:33.249435 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:33.249514 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:33.251692 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:33.251700 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:33.251712 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:33.251734 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:33.253524 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:33.253925 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:33.253930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:33.253931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:33.254005 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:33.254229 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:33.254231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:33.254232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:33.254450 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:33.254630 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:33.254750 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 18:00:33.254911 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260923 18:00:33.259130 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:33.258089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37238 (local address 127.15.139.129:41657)
0923 18:00:33.258225 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:33.258229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:33.258344 (+ 115us) server_negotiation.cc:407] Connection header received
0923 18:00:33.258482 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:33.258485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:33.258561 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:33.258651 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:33.259029 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37238: BlockingRecv error: recv got EOF from 127.0.0.1:37238 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260923 18:00:33.259311 15918 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/krb5cc' of type FILE
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:33.260458 15918 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:33.261528 15918 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.129@KRBTEST.COM
I20260923 18:00:33.270323 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:33.262284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37252 (local address 127.15.139.129:41657)
0923 18:00:33.262405 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:33.262409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:33.262547 (+ 138us) server_negotiation.cc:407] Connection header received
0923 18:00:33.262694 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:33.262697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:33.262740 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:33.262831 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:33.263327 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:33.263912 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:33.264726 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:33.264979 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:33.265090 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:33.267285 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:33.267289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:33.267292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:33.267321 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:33.268708 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:33.269175 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:33.269180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:33.269182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:33.269259 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:33.269539 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:33.269544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:33.269546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:33.269711 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:33.269843 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:33.270070 (+ 227us) server_negotiation.cc:299] Negotiation successful
0923 18:00:33.270181 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260923 18:00:33.273140 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:33.272097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37260 (local address 127.15.139.129:41657)
0923 18:00:33.272216 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:33.272220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:33.272343 (+ 123us) server_negotiation.cc:407] Connection header received
0923 18:00:33.272472 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:33.272475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:33.272517 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:33.272599 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:33.272988 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37260: BlockingRecv error: recv got EOF from 127.0.0.1:37260 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260923 18:00:33.273272 15918 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790186398863621-15918-0/krb5kdc/krb5cc' of type FILE
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:33.274266 15918 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:33.275123 15918 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.129@KRBTEST.COM
I20260923 18:00:33.283986 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:33.275956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37272 (local address 127.15.139.129:41657)
0923 18:00:33.276074 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:33.276078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:33.276200 (+ 122us) server_negotiation.cc:407] Connection header received
0923 18:00:33.276347 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:33.276350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:33.276382 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:33.276464 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:33.276987 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:33.277493 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:33.278348 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:33.278569 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:33.278677 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:33.280879 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:33.280888 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:33.280890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:33.280910 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:33.282194 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:33.282706 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:33.282710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:33.282712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:33.282795 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:33.283157 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:33.283161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:33.283163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:33.283328 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:33.283465 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:33.283730 (+ 265us) server_negotiation.cc:299] Negotiation successful
0923 18:00:33.283834 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 18:00:33.284960 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 21575
I20260923 18:00:33.290839 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 21709
I20260923 18:00:33.296348 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 21843
I20260923 18:00:33.301479 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 21481
2026-09-23T18:00:33Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2267 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21992](info): set up 2 sockets
Sep 23 18:00:33 dist-test-slave-vr57 krb5kdc[21992](info): commencing operation
krb5kdc: starting...
W20260923 18:00:34.408396 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:34 dist-test-slave-vr57 krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:34Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.190.keytab.
Entry for principal oryx/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.190.keytab.
I20260923 18:00:34.535490 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:41269
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.190.keytab
--principal=oryx/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:41269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:34.647706 22005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:34.647992 22005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:34.648036 22005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:34.652159 22005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:34.652225 22005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:34.652245 22005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:34.652263 22005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:34.652311 22005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:34.657832 22005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:41269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.190.keytab
--principal=oryx/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:41269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:34.658948 22005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:34.660084 22005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:34.668744 22011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:34.668749 22010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:34.668772 22013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:34.669137 22005 server_base.cc:1034] running on GCE node
I20260923 18:00:34.669775 22005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:34.670899 22005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:34.672093 22005 hybrid_clock.cc:648] HybridClock initialized: now 1790186434672077 us; error 36 us; skew 500 ppm
Sep 23 18:00:34 dist-test-slave-vr57 krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186434, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:34.674863 22005 init.cc:377] Logged in from keytab as oryx/127.15.139.190@KRBTEST.COM (short username oryx)
I20260923 18:00:34.675848 22005 webserver.cc:460] Webserver started at http://127.15.139.190:40955/ using document root <none> and password file <none>
I20260923 18:00:34.676376 22005 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:34.676436 22005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:34.676615 22005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:34.678629 22005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "be70c9d1f51f405c8c9e59aa78004870"
format_stamp: "Formatted at 2026-09-23 18:00:34 on dist-test-slave-vr57"
server_key: "a6d070a2acf5bd2c8eceeebdffdd2a65"
server_key_iv: "2f1dfd6637feda6f3da1ae69c22eb485"
server_key_version: "encryptionkey@0"
I20260923 18:00:34.679134 22005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "be70c9d1f51f405c8c9e59aa78004870"
format_stamp: "Formatted at 2026-09-23 18:00:34 on dist-test-slave-vr57"
server_key: "a6d070a2acf5bd2c8eceeebdffdd2a65"
server_key_iv: "2f1dfd6637feda6f3da1ae69c22eb485"
server_key_version: "encryptionkey@0"
I20260923 18:00:34.682789 22005 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260923 18:00:34.685211 22020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:34.686507 22005 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:34.686697 22005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "be70c9d1f51f405c8c9e59aa78004870"
format_stamp: "Formatted at 2026-09-23 18:00:34 on dist-test-slave-vr57"
server_key: "a6d070a2acf5bd2c8eceeebdffdd2a65"
server_key_iv: "2f1dfd6637feda6f3da1ae69c22eb485"
server_key_version: "encryptionkey@0"
I20260923 18:00:34.686813 22005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:34.723814 22005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:34.727453 22005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:34.727643 22005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:34.735312 22005 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:41269
I20260923 18:00:34.735657 22072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:41269 every 8 connection(s)
I20260923 18:00:34.736487 22005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:34.739441 22073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:34.742437 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 22005
I20260923 18:00:34.742627 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:34.742933 15918 external_mini_cluster.cc:1428] Setting key 8cfa5a8886df9706a4e4c497d5f7004f
I20260923 18:00:34.745726 22073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870: Bootstrap starting.
I20260923 18:00:34.748447 22073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:34.749208 22073 log.cc:824] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:34.751348 22073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870: No bootstrap required, opened a new log
Sep 23 18:00:34 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:34.755276 22073 raft_consensus.cc:348] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } }
I20260923 18:00:34.755548 22073 raft_consensus.cc:374] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:34.755625 22073 raft_consensus.cc:729] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be70c9d1f51f405c8c9e59aa78004870, State: Initialized, Role: FOLLOWER
I20260923 18:00:34.756127 22073 consensus_queue.cc:260] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } }
I20260923 18:00:34.756287 22073 raft_consensus.cc:388] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:34.756345 22073 raft_consensus.cc:482] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:34.756438 22073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:34.757618 22073 raft_consensus.cc:504] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } }
I20260923 18:00:34.757984 22073 leader_election.cc:302] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be70c9d1f51f405c8c9e59aa78004870; no voters:
I20260923 18:00:34.758320 22073 leader_election.cc:288] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:34.758435 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:34.744553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42116 (local address 127.15.139.190:41269)
0923 18:00:34.745007 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:34.745026 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:34.745075 (+ 49us) server_negotiation.cc:407] Connection header received
0923 18:00:34.745687 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:34.745704 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:34.746104 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:34.746389 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:34.747377 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:34.748531 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:34.749431 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:34.750105 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:34.750271 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:34.753033 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:34.753038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:34.753052 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:34.753079 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:34.754943 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:34.755462 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:34.755472 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:34.755476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:34.755666 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:34.756018 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:34.756022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:34.756024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:34.756586 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:34.756801 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:34.757120 (+ 319us) server_negotiation.cc:299] Negotiation successful
0923 18:00:34.757436 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":172,"threads_started":1}
I20260923 18:00:34.758445 22078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:34.758692 22078 raft_consensus.cc:686] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 1 LEADER]: Becoming Leader. State: Replica: be70c9d1f51f405c8c9e59aa78004870, State: Running, Role: LEADER
I20260923 18:00:34.759063 22078 consensus_queue.cc:237] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } }
I20260923 18:00:34.759591 22073 sys_catalog.cc:564] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:34.760376 22080 sys_catalog.cc:455] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } }
I20260923 18:00:34.760536 22080 sys_catalog.cc:458] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:34.760828 22079 sys_catalog.cc:455] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be70c9d1f51f405c8c9e59aa78004870. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } }
I20260923 18:00:34.760941 22079 sys_catalog.cc:458] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:34.760983 22087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:34.764506 22087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:34.770886 22087 catalog_manager.cc:1347] Generated new cluster ID: c84c2c9cb9bd495391ee03b26f00e237
I20260923 18:00:34.771087 22087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:34.795074 22087 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:34.796242 22087 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:34.804704 22087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870: Generated new TSK 0
I20260923 18:00:34.805440 22087 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab.
Entry for principal oryx/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab.
I20260923 18:00:34.838908 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:41269
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab
--principal=oryx/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:34.954605 22099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:34.954867 22099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:34.954911 22099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:34.959329 22099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:34.959421 22099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:34.959487 22099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:34.965493 22099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab
--principal=oryx/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:34.966615 22099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:34.967754 22099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:34.977778 22105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:34.977835 22104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:34.978166 22099 server_base.cc:1034] running on GCE node
W20260923 18:00:34.978356 22107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:34.978950 22099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:34.979977 22099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:34.981216 22099 hybrid_clock.cc:648] HybridClock initialized: now 1790186434981181 us; error 54 us; skew 500 ppm
Sep 23 18:00:34 dist-test-slave-vr57 krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186434, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:34.985038 22099 init.cc:377] Logged in from keytab as oryx/127.15.139.129@KRBTEST.COM (short username oryx)
I20260923 18:00:34.986492 22099 webserver.cc:460] Webserver started at http://127.15.139.129:33561/ using document root <none> and password file <none>
I20260923 18:00:34.987329 22099 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:34.987450 22099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:34.987824 22099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:34.990471 22099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "e53e7fbff400481f8979b8602fcd6a9b"
format_stamp: "Formatted at 2026-09-23 18:00:34 on dist-test-slave-vr57"
server_key: "2a00457f1d0962b6423668c546e0e090"
server_key_iv: "478e68ffff060bec1ccbc811f7981866"
server_key_version: "encryptionkey@0"
I20260923 18:00:34.991147 22099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "e53e7fbff400481f8979b8602fcd6a9b"
format_stamp: "Formatted at 2026-09-23 18:00:34 on dist-test-slave-vr57"
server_key: "2a00457f1d0962b6423668c546e0e090"
server_key_iv: "478e68ffff060bec1ccbc811f7981866"
server_key_version: "encryptionkey@0"
I20260923 18:00:34.995657 22099 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:34.998471 22114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:34.999876 22099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:35.000025 22099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "e53e7fbff400481f8979b8602fcd6a9b"
format_stamp: "Formatted at 2026-09-23 18:00:34 on dist-test-slave-vr57"
server_key: "2a00457f1d0962b6423668c546e0e090"
server_key_iv: "478e68ffff060bec1ccbc811f7981866"
server_key_version: "encryptionkey@0"
I20260923 18:00:35.000164 22099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:35.016983 22099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:35.020464 22099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:35.020720 22099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:35.021427 22099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:35.022426 22099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:35.022508 22099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:35.022591 22099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:35.022634 22099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:35.032585 22099 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:46199
I20260923 18:00:35.032603 22227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:46199 every 8 connection(s)
I20260923 18:00:35.033613 22099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:35.036091 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 22099
I20260923 18:00:35.036231 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:35.036648 15918 external_mini_cluster.cc:1428] Setting key 002a6f553723489c681c42ef6ccacaba
Sep 23 18:00:35 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186434, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.15.139.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.130@KRBTEST.COM" created.
I20260923 18:00:35.050207 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.035616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:35587 (local address 127.15.139.190:41269)
0923 18:00:35.035797 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:35.035802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:35.036969 (+ 1167us) server_negotiation.cc:407] Connection header received
0923 18:00:35.038907 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:35.038909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:35.038952 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:35.039036 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:35.040814 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.041427 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.042819 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.043088 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.043212 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.046115 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:35.046118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:35.046121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:35.046146 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:35.047863 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.048394 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.048397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.048398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.048546 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.048978 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.048982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.048984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.049191 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:35.049310 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:35.049859 (+ 549us) server_negotiation.cc:299] Negotiation successful
0923 18:00:35.050008 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 18:00:35.051126 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.036047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.129:35587)
0923 18:00:35.036762 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:35.036832 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:35.038630 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:35.039269 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:35.039283 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:35.039939 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:35.040649 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.040666 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.041573 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.041576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.042681 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.042692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.043271 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.043276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.043470 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.044253 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:35.044272 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:35.045874 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:35.047990 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.047995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.048008 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.048290 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.048654 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.048659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.048661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.048856 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.049308 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:35.049316 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:35.049590 (+ 274us) client_negotiation.cc:769] Sending connection context
0923 18:00:35.049797 (+ 207us) client_negotiation.cc:240] Negotiation successful
0923 18:00:35.050174 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":170,"threads_started":1}
I20260923 18:00:35.052716 22228 heartbeater.cc:347] Connected to a master server at 127.15.139.190:41269
I20260923 18:00:35.053005 22228 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:35.053550 22228 heartbeater.cc:511] Master 127.15.139.190:41269 requested a full tablet report, sending...
I20260923 18:00:35.055341 22037 ts_manager.cc:194] Registered new tserver with Master: e53e7fbff400481f8979b8602fcd6a9b (127.15.139.129:46199)
I20260923 18:00:35.056996 22037 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:35587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab.
Entry for principal oryx/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab.
I20260923 18:00:35.066524 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:41269
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab
--principal=oryx/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:35.191187 22233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:35.191488 22233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:35.191608 22233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:35.196125 22233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:35.196192 22233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:35.196252 22233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:35.201573 22233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab
--principal=oryx/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:35.202726 22233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:35.203800 22233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:35.213682 22241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:35.213794 22238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:35.213697 22239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:35.214201 22233 server_base.cc:1034] running on GCE node
I20260923 18:00:35.214733 22233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:35.215442 22233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:35.216663 22233 hybrid_clock.cc:648] HybridClock initialized: now 1790186435216626 us; error 51 us; skew 500 ppm
Sep 23 18:00:35 dist-test-slave-vr57 krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186435, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:35.219609 22233 init.cc:377] Logged in from keytab as oryx/127.15.139.130@KRBTEST.COM (short username oryx)
I20260923 18:00:35.220484 22233 webserver.cc:460] Webserver started at http://127.15.139.130:39691/ using document root <none> and password file <none>
I20260923 18:00:35.221107 22233 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:35.221204 22233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:35.221447 22233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:35.223639 22233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "b844678d1ab24411857fc79ae7010d2c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "6c496c5e6119a4968ace6483ee55a6fc"
server_key_iv: "7c03ea1e6f5b8902f5b88aa2cc4dee8d"
server_key_version: "encryptionkey@0"
I20260923 18:00:35.224267 22233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "b844678d1ab24411857fc79ae7010d2c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "6c496c5e6119a4968ace6483ee55a6fc"
server_key_iv: "7c03ea1e6f5b8902f5b88aa2cc4dee8d"
server_key_version: "encryptionkey@0"
I20260923 18:00:35.228274 22233 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:00:35.231343 22248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:35.232800 22233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:00:35.232985 22233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "b844678d1ab24411857fc79ae7010d2c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "6c496c5e6119a4968ace6483ee55a6fc"
server_key_iv: "7c03ea1e6f5b8902f5b88aa2cc4dee8d"
server_key_version: "encryptionkey@0"
I20260923 18:00:35.233129 22233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:35.265511 22233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:35.269230 22233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:35.269465 22233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:35.270202 22233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:35.271106 22233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:35.271180 22233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:35.271255 22233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:35.271310 22233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:35.280416 22233 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:36899
I20260923 18:00:35.280444 22361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:36899 every 8 connection(s)
I20260923 18:00:35.281420 22233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:35.283900 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 22233
I20260923 18:00:35.284097 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:35.284487 15918 external_mini_cluster.cc:1428] Setting key 466346744b338ebca0e44ea9c47f8cd6
Sep 23 18:00:35 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186435, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.15.139.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.131@KRBTEST.COM" created.
I20260923 18:00:35.298358 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.283665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:52191 (local address 127.15.139.190:41269)
0923 18:00:35.283830 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:35.283834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:35.284900 (+ 1066us) server_negotiation.cc:407] Connection header received
0923 18:00:35.286597 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:35.286600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:35.286655 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:35.286732 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:35.288361 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.288894 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.290819 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.291180 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.291304 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.294278 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:35.294284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:35.294287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:35.294324 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:35.295926 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.296621 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.296624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.296625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.296699 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.297014 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.297016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.297017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.297258 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:35.297335 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:35.297937 (+ 602us) server_negotiation.cc:299] Negotiation successful
0923 18:00:35.298095 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 18:00:35.299396 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.284039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.130:52191)
0923 18:00:35.284722 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:35.284780 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:35.286356 (+ 1576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:35.286914 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:35.286927 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:35.287538 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:35.288211 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.288225 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.289027 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.289033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.290670 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.290687 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.291348 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.291351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.291613 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.292269 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:35.292285 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:35.294093 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:35.296102 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.296107 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.296120 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.296508 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.296780 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.296782 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.296783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.296921 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.297348 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:35.297382 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:35.297641 (+ 259us) client_negotiation.cc:769] Sending connection context
0923 18:00:35.297867 (+ 226us) client_negotiation.cc:240] Negotiation successful
0923 18:00:35.298277 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":123,"threads_started":1}
I20260923 18:00:35.300822 22362 heartbeater.cc:347] Connected to a master server at 127.15.139.190:41269
I20260923 18:00:35.301096 22362 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:35.301637 22362 heartbeater.cc:511] Master 127.15.139.190:41269 requested a full tablet report, sending...
I20260923 18:00:35.302775 22037 ts_manager.cc:194] Registered new tserver with Master: b844678d1ab24411857fc79ae7010d2c (127.15.139.130:36899)
I20260923 18:00:35.303372 22037 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:52191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab.
Entry for principal oryx/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab.
I20260923 18:00:35.318156 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:41269
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab
--principal=oryx/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:35.432309 22367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:35.432578 22367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:35.432621 22367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:35.436726 22367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:35.436787 22367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:35.436846 22367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:35.442116 22367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab
--principal=oryx/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:35.443115 22367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:35.444160 22367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:35.453857 22372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:35.453851 22373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:35.454030 22375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:35.453913 22367 server_base.cc:1034] running on GCE node
I20260923 18:00:35.454613 22367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:35.455291 22367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:35.456475 22367 hybrid_clock.cc:648] HybridClock initialized: now 1790186435456437 us; error 50 us; skew 500 ppm
Sep 23 18:00:35 dist-test-slave-vr57 krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186435, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:35.459486 22367 init.cc:377] Logged in from keytab as oryx/127.15.139.131@KRBTEST.COM (short username oryx)
I20260923 18:00:35.460675 22367 webserver.cc:460] Webserver started at http://127.15.139.131:39279/ using document root <none> and password file <none>
I20260923 18:00:35.461325 22367 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:35.461400 22367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:35.461622 22367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:35.463831 22367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "f248c4fde5a24ad28b187849e4eb225c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "033aa4ee648887e55b742ed4de2f5e05"
server_key_iv: "d716c7cb9f1848feb7662ba0c1f1410a"
server_key_version: "encryptionkey@0"
I20260923 18:00:35.464401 22367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "f248c4fde5a24ad28b187849e4eb225c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "033aa4ee648887e55b742ed4de2f5e05"
server_key_iv: "d716c7cb9f1848feb7662ba0c1f1410a"
server_key_version: "encryptionkey@0"
I20260923 18:00:35.468349 22367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260923 18:00:35.470746 22382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:35.472138 22367 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260923 18:00:35.472272 22367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "f248c4fde5a24ad28b187849e4eb225c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "033aa4ee648887e55b742ed4de2f5e05"
server_key_iv: "d716c7cb9f1848feb7662ba0c1f1410a"
server_key_version: "encryptionkey@0"
I20260923 18:00:35.472388 22367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:35.493717 22367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:35.497275 22367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:35.497498 22367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:35.498123 22367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:35.499081 22367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:35.499146 22367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:35.499233 22367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:35.499269 22367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:35.508587 22367 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:45875
I20260923 18:00:35.508623 22495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:45875 every 8 connection(s)
I20260923 18:00:35.509470 22367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:35.515568 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 22367
I20260923 18:00:35.515713 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:00:35.515991 15918 external_mini_cluster.cc:1428] Setting key 29108ec44ea2adcf715e04fef405742f
Sep 23 18:00:35 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186435, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:35.525383 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.511653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:46545 (local address 127.15.139.190:41269)
0923 18:00:35.511795 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:35.511799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:35.512755 (+ 956us) server_negotiation.cc:407] Connection header received
0923 18:00:35.514149 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:35.514152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:35.514209 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:35.514319 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:35.515612 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.516420 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.518062 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.518461 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.518579 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.521803 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:35.521806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:35.521808 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:35.521832 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:35.523323 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.523928 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.523931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.523932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.524015 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.524365 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.524367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.524368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.524544 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:35.524621 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:35.525158 (+ 537us) server_negotiation.cc:299] Negotiation successful
0923 18:00:35.525264 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 18:00:35.526305 22498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.512128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.131:46545)
0923 18:00:35.512604 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:35.512648 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:35.513928 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:35.514492 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:35.514499 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:35.514858 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:35.515409 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.515432 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.516555 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.516559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.517930 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.517939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.518760 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.518767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.519011 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.519915 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:35.519937 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:35.521620 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:35.523462 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.523468 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.523480 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.523800 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.524142 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.524144 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.524145 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.524277 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.524633 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:35.524637 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:35.524873 (+ 236us) client_negotiation.cc:769] Sending connection context
0923 18:00:35.525091 (+ 218us) client_negotiation.cc:240] Negotiation successful
0923 18:00:35.525374 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":121,"threads_started":1}
I20260923 18:00:35.527596 22496 heartbeater.cc:347] Connected to a master server at 127.15.139.190:41269
I20260923 18:00:35.527817 22496 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:35.528292 22496 heartbeater.cc:511] Master 127.15.139.190:41269 requested a full tablet report, sending...
I20260923 18:00:35.529282 22037 ts_manager.cc:194] Registered new tserver with Master: f248c4fde5a24ad28b187849e4eb225c (127.15.139.131:45875)
I20260923 18:00:35.529865 22037 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:46545
I20260923 18:00:35.530791 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.139.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.189.keytab.
Entry for principal oryx/127.15.139.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.189.keytab.
I20260923 18:00:35.558341 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.189:44001
--webserver_interface=127.15.139.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.189.keytab
--principal=oryx/127.15.139.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:41269,127.15.139.189:44001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:35.670243 22501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:35.670502 22501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:35.670542 22501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:35.674482 22501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:35.674539 22501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:35.674559 22501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:35.674576 22501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:35.674615 22501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:35.680040 22501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:41269,127.15.139.189:44001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.189.keytab
--principal=oryx/127.15.139.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.189:44001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.139.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
W20260923 18:00:35.680255 22501 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.139.190:41269,127.15.139.189:44001'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260923 18:00:35.681113 22501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:35.682160 22501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:35.690856 22509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:35.690856 22506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:35.690917 22501 server_base.cc:1034] running on GCE node
W20260923 18:00:35.691181 22507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:35.691869 22501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:35.692965 22501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:35.694135 22501 hybrid_clock.cc:648] HybridClock initialized: now 1790186435694114 us; error 31 us; skew 500 ppm
Sep 23 18:00:35 dist-test-slave-vr57 krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186435, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:35.697320 22501 init.cc:377] Logged in from keytab as oryx/127.15.139.189@KRBTEST.COM (short username oryx)
I20260923 18:00:35.698210 22501 webserver.cc:460] Webserver started at http://127.15.139.189:35685/ using document root <none> and password file <none>
I20260923 18:00:35.698719 22501 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:35.698777 22501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:35.698951 22501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:35.701011 22501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data/instance:
uuid: "460156afb37e44d5a0c97e63447c0c7c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "3ced12333bfb2ed8c8cb4095e656a611"
server_key_iv: "9d0607c13d0efa800e7522f4437c4d86"
server_key_version: "encryptionkey@0"
I20260923 18:00:35.701507 22501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/wal/instance:
uuid: "460156afb37e44d5a0c97e63447c0c7c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "3ced12333bfb2ed8c8cb4095e656a611"
server_key_iv: "9d0607c13d0efa800e7522f4437c4d86"
server_key_version: "encryptionkey@0"
I20260923 18:00:35.705281 22501 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:35.707686 22516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:35.708926 22501 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:00:35.709045 22501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/wal
uuid: "460156afb37e44d5a0c97e63447c0c7c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "3ced12333bfb2ed8c8cb4095e656a611"
server_key_iv: "9d0607c13d0efa800e7522f4437c4d86"
server_key_version: "encryptionkey@0"
I20260923 18:00:35.709168 22501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:35.741567 22501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:35.744993 22501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:35.745216 22501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:35.752314 22501 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.189:44001
I20260923 18:00:35.752633 22568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.189:44001 every 8 connection(s)
I20260923 18:00:35.753432 22501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
I20260923 18:00:35.755467 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 22501
I20260923 18:00:35.756739 22569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:35.761035 22569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 18:00:35 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.139.189@KRBTEST.COM
Sep 23 18:00:35 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186435, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.189@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:35.771294 22572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.757154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33032 (local address 127.15.139.189:44001)
0923 18:00:35.757625 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:35.757641 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:35.757676 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:00:35.758318 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:35.758334 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:35.758664 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:35.759076 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:35.759920 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.761040 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.761937 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.762395 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.762561 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.765449 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:35.765455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:35.765466 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:35.765496 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:35.767905 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.768447 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.768451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.768455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.768563 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.768873 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.768876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.768877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.769394 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:35.769630 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:35.769962 (+ 332us) server_negotiation.cc:299] Negotiation successful
0923 18:00:35.770271 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":154,"threads_started":1}
I20260923 18:00:35.772375 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.762749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42126 (local address 127.15.139.190:41269)
0923 18:00:35.763228 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:35.763235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:35.763252 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:00:35.763300 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:35.763305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:35.763357 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:35.763451 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:35.764231 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.764784 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.765972 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.766193 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.766302 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.768849 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:35.768853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:35.768859 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:35.768887 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:35.770521 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.771002 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.771005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.771007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.771093 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.771354 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.771358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.771363 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.771676 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:35.771775 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:35.772107 (+ 332us) server_negotiation.cc:299] Negotiation successful
0923 18:00:35.772221 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 18:00:35.772558 22573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.762699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42126)
0923 18:00:35.763031 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:35.763077 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:35.763181 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:35.763550 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:35.763556 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:35.763887 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:35.764093 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.764099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.764920 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.764924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.765857 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.765866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.766303 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.766306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.766426 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.766904 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:35.766925 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:35.768701 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:35.770679 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.770687 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.770691 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.770907 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.771184 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.771186 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.771188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.771266 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.771808 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:35.771814 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:35.771912 (+ 98us) client_negotiation.cc:769] Sending connection context
0923 18:00:35.772037 (+ 125us) client_negotiation.cc:240] Negotiation successful
0923 18:00:35.772219 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":88,"threads_started":1}
I20260923 18:00:35.773813 22569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 18:00:35 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186435, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.189@KRBTEST.COM for oryx/127.15.139.189@KRBTEST.COM
I20260923 18:00:35.782287 22573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.774355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44001 (local address 127.0.0.1:33048)
0923 18:00:35.774474 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:35.774491 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:35.774553 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:35.774783 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:35.774785 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:35.774917 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:35.775043 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.775047 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.776192 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.776195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.776844 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.776853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.777280 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.777283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.777361 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.777794 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:35.777805 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:35.779045 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:35.780924 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.780927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.780928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.781148 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.781430 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.781431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.781433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.781510 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.781826 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:35.781828 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:35.781898 (+ 70us) client_negotiation.cc:769] Sending connection context
0923 18:00:35.781977 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 18:00:35.782126 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260923 18:00:35.782302 22572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.774410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.15.139.189:44001)
0923 18:00:35.774521 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:35.774524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:35.774535 (+ 11us) server_negotiation.cc:407] Connection header received
0923 18:00:35.774629 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:35.774630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:35.774680 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:35.774766 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:35.775140 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.776058 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.776944 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.777168 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.777247 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.779164 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:35.779167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:35.779168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:35.779187 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:35.780666 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.781266 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.781269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.781270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.781342 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.781590 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.781592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.781593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.781748 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:35.781820 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:35.782029 (+ 209us) server_negotiation.cc:299] Negotiation successful
0923 18:00:35.782184 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":44}
I20260923 18:00:35.785322 22569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: Bootstrap starting.
I20260923 18:00:35.787596 22569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:35.788273 22569 log.cc:824] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:35.790148 22569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: No bootstrap required, opened a new log
I20260923 18:00:35.793166 22569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } }
I20260923 18:00:35.793365 22569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:35.793398 22569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460156afb37e44d5a0c97e63447c0c7c, State: Initialized, Role: FOLLOWER
I20260923 18:00:35.793807 22569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } }
I20260923 18:00:35.794232 22575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } } }
I20260923 18:00:35.794368 22575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:00:35.794963 22569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: configured and running, proceeding with master startup.
W20260923 18:00:35.799722 22586 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:00:35.800025 22586 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:00:35.802047 22501 master_runner.cc:428] Detected that this master 460156afb37e44d5a0c97e63447c0c7c is joining an existing cluster
I20260923 18:00:35.802111 22501 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.139.189:44001
I20260923 18:00:35.814565 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.804526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42132 (local address 127.15.139.190:41269)
0923 18:00:35.804649 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:35.804654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:35.805397 (+ 743us) server_negotiation.cc:407] Connection header received
0923 18:00:35.805515 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:35.805519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:35.805594 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:35.805679 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:35.806277 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.807155 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.808249 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.808504 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.808643 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.810534 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:35.810541 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:35.810546 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:35.810579 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:35.812543 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.813208 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.813212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.813214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.813315 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.813626 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.813633 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.813637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.813857 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:35.813981 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:35.814277 (+ 296us) server_negotiation.cc:299] Negotiation successful
0923 18:00:35.814401 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 18:00:35.814584 22593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.804758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42132)
0923 18:00:35.805296 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:35.805311 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:35.805416 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:35.805817 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:35.805821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:35.806040 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:35.806148 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.806154 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.807281 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.807284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.808104 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.808114 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.808656 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.808660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.808763 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.809856 (+ 1093us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:35.809889 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:35.810353 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:35.812817 (+ 2464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.812820 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.812823 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.813114 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.813427 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.813430 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.813432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.813525 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.813981 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:35.813986 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:35.814091 (+ 105us) client_negotiation.cc:769] Sending connection context
0923 18:00:35.814206 (+ 115us) client_negotiation.cc:240] Negotiation successful
0923 18:00:35.814355 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":19,"thread_start_us":111,"threads_started":1}
I20260923 18:00:35.815155 22572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.804758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.15.139.189:44001)
0923 18:00:35.804887 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:35.804892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:35.805178 (+ 286us) server_negotiation.cc:407] Connection header received
0923 18:00:35.805464 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:35.805474 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:35.805523 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:35.805613 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:35.807308 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.808189 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.809412 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.809825 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.809948 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.811298 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:35.811304 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:35.811307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:35.811339 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:35.813561 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.813997 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.814001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.814003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.814066 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.814302 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.814305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.814306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.814476 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:35.814541 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:35.814811 (+ 270us) server_negotiation.cc:299] Negotiation successful
0923 18:00:35.814912 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":59}
I20260923 18:00:35.815773 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.804598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44001 (local address 127.0.0.1:33052)
0923 18:00:35.805067 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:35.805084 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:35.805313 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:35.805661 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:35.805668 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:35.805961 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:35.806140 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.806148 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.808324 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.808328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.809274 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.809285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.809967 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.809971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.810120 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.810705 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:35.810719 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:35.811145 (+ 426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:35.813674 (+ 2529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.813677 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.813679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.813892 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.814154 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.814157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.814159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.814224 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.814596 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:35.814598 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:35.814667 (+ 69us) client_negotiation.cc:769] Sending connection context
0923 18:00:35.815346 (+ 679us) client_negotiation.cc:240] Negotiation successful
0923 18:00:35.815584 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":22,"thread_start_us":105,"threads_started":1}
Sep 23 18:00:35 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186434, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.190@KRBTEST.COM for oryx/127.15.139.189@KRBTEST.COM
I20260923 18:00:35.830964 22572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.820175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33066 (local address 127.15.139.189:44001)
0923 18:00:35.820291 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:35.820295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:35.820698 (+ 403us) server_negotiation.cc:407] Connection header received
0923 18:00:35.820889 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:35.820892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:35.820946 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:35.821040 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:35.821738 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.822552 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.823549 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.823896 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.824074 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.826803 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:35.826812 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:35.826816 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:35.826838 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:35.828832 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.829457 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.829464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.829466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.829557 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.829873 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.829877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.829879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.830120 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:35.830235 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:35.830625 (+ 390us) server_negotiation.cc:299] Negotiation successful
0923 18:00:35.830776 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 18:00:35.830979 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.820157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44001 (local address 127.0.0.1:33066)
0923 18:00:35.820567 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:35.820620 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:35.820753 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:35.821090 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:35.821095 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:35.821519 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:35.821633 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.821639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.822666 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.822669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.823411 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.823421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.824128 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.824132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.824272 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.824996 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:35.825014 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:35.826650 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:35.829020 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.829032 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.829038 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.829313 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.829662 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.829666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.829668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.829771 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.830251 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:35.830261 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:35.830404 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 18:00:35.830569 (+ 165us) client_negotiation.cc:240] Negotiation successful
0923 18:00:35.830791 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":129,"threads_started":1}
I20260923 18:00:35.832669 22037 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.139.189:44001: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c"
member_type: NON_VOTER
last_known_addr {
host: "127.15.139.189"
port: 44001
}
attrs {
promote: true
}
}
dest_uuid: "be70c9d1f51f405c8c9e59aa78004870"
cas_config_opid_index: -1
I20260923 18:00:35.833397 22037 consensus_queue.cc:237] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } }
I20260923 18:00:35.835336 22597 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 460156afb37e44d5a0c97e63447c0c7c (127.15.139.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } }
I20260923 18:00:35.836440 22543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 0 FOLLOWER]: Advancing to term 1
WARNING: no policy specified for oryx/127.15.139.188@KRBTEST.COM; defaulting to no policy
I20260923 18:00:35.837926 22597 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.139.189:44001
I20260923 18:00:35.838097 22543 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 FOLLOWER]: Refusing update from remote peer be70c9d1f51f405c8c9e59aa78004870: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260923 18:00:35.838532 22598 sys_catalog.cc:455] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } }
I20260923 18:00:35.838636 22598 sys_catalog.cc:458] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:35.838739 22501 master.cc:537] Master@127.15.139.189:44001 shutting down...
I20260923 18:00:35.839315 22597 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.001s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.188@KRBTEST.COM" created.
W20260923 18:00:35.842021 22022 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260923 18:00:35.843338 22022 consensus_peers.cc:487] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 -> Peer 460156afb37e44d5a0c97e63447c0c7c (127.15.139.189:44001): Couldn't send request to peer 460156afb37e44d5a0c97e63447c0c7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 18:00:35.844007 22501 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:35.844310 22501 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:35.844446 22501 tablet_replica.cc:330] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: stopping tablet replica
I20260923 18:00:35.849772 22501 master.cc:552] Master@127.15.139.189:44001 shutdown complete.
I20260923 18:00:35.849853 22501 master_runner.cc:305] Clearing existing system tablet
I20260923 18:00:35.850116 22501 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:35.850194 22501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:35.851486 22501 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 18:00:35.852982 22604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:35.853190 22501 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:35.853341 22501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 18:00:35.853387 22501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/wal
uuid: "460156afb37e44d5a0c97e63447c0c7c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "3ced12333bfb2ed8c8cb4095e656a611"
server_key_iv: "9d0607c13d0efa800e7522f4437c4d86"
server_key_version: "encryptionkey@0"
I20260923 18:00:35.856403 22501 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.188.keytab.
Entry for principal oryx/127.15.139.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.188.keytab.
I20260923 18:00:35.858879 22501 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 18:00:35.858906 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.188:37763
--webserver_interface=127.15.139.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.188.keytab
--principal=oryx/127.15.139.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:41269,127.15.139.189:44001,127.15.139.188:37763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:00:35.859163 22501 log.cc:1196] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260923 18:00:35.859570 22501 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: Deleting consensus metadata
I20260923 18:00:35.859760 22501 master_runner.cc:315] Copying system tablet from 127.15.139.190:41269
I20260923 18:00:35.861264 22501 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.139.190:41269
I20260923 18:00:35.869925 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.861879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42134)
0923 18:00:35.862189 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:35.862207 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:35.862343 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:35.862616 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:35.862619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:35.862821 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:35.862928 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.862934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.863971 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.863978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.864841 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:35.864850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.865341 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.865345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:35.865449 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.866016 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:35.866029 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:35.866369 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:35.868292 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.868297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.868302 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.868521 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.868830 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:35.868833 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:35.868835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:35.868922 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:35.869341 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:35.869344 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:35.869455 (+ 111us) client_negotiation.cc:769] Sending connection context
0923 18:00:35.869569 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 18:00:35.869726 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":87,"threads_started":1}
I20260923 18:00:35.869908 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:35.861960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42134 (local address 127.15.139.190:41269)
0923 18:00:35.862098 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:35.862104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:35.862293 (+ 189us) server_negotiation.cc:407] Connection header received
0923 18:00:35.862450 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:35.862453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:35.862502 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:35.862598 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:35.863050 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.863845 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.864973 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:35.865209 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:35.865323 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:35.866537 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:35.866540 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:35.866543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:35.866573 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:35.868159 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.868635 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.868640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.868642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.868727 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:35.869039 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:35.869042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:35.869044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:35.869233 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:35.869351 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:35.869624 (+ 273us) server_negotiation.cc:299] Negotiation successful
0923 18:00:35.869739 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 18:00:35.870666 22057 tablet_copy_service.cc:140] P be70c9d1f51f405c8c9e59aa78004870: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 460156afb37e44d5a0c97e63447c0c7c ({username='oryx', principal='oryx/127.15.139.189@KRBTEST.COM'} at 127.0.0.1:42134)
I20260923 18:00:35.870854 22057 tablet_copy_service.cc:161] P be70c9d1f51f405c8c9e59aa78004870: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 460156afb37e44d5a0c97e63447c0c7c at {username='oryx', principal='oryx/127.15.139.189@KRBTEST.COM'} at 127.0.0.1:42134: session id = 460156afb37e44d5a0c97e63447c0c7c-00000000000000000000000000000000
I20260923 18:00:35.872372 22057 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 18:00:35.873955 22501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 18:00:35.876117 22501 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: tablet copy: Starting download of 0 data blocks...
I20260923 18:00:35.876317 22501 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: tablet copy: Starting download of 1 WAL segments...
I20260923 18:00:35.878088 22501 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 18:00:35.879735 22057 tablet_copy_service.cc:342] P be70c9d1f51f405c8c9e59aa78004870: Request end of tablet copy session 460156afb37e44d5a0c97e63447c0c7c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.139.189@KRBTEST.COM'} at 127.0.0.1:42134
I20260923 18:00:35.879871 22057 tablet_copy_service.cc:433] P be70c9d1f51f405c8c9e59aa78004870: ending tablet copy session 460156afb37e44d5a0c97e63447c0c7c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 460156afb37e44d5a0c97e63447c0c7c
I20260923 18:00:35.887288 22501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:35.888974 22615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:35.889050 22612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:35.889329 22501 server_base.cc:1034] running on GCE node
W20260923 18:00:35.891676 22613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:35.891911 22501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:35.892333 22501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:35.893502 22501 hybrid_clock.cc:648] HybridClock initialized: now 1790186435893519 us; error 76 us; skew 500 ppm
Sep 23 18:00:35 dist-test-slave-vr57 krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186435, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:35.895170 22501 init.cc:377] Logged in from keytab as oryx/127.15.139.189@KRBTEST.COM (short username oryx)
I20260923 18:00:35.895797 22501 webserver.cc:460] Webserver started at http://127.15.139.189:44583/ using document root <none> and password file <none>
I20260923 18:00:35.896031 22501 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:35.896092 22501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:35.897316 22501 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:35.898356 22622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:35.898608 22501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:35.898713 22501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/wal
uuid: "460156afb37e44d5a0c97e63447c0c7c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "3ced12333bfb2ed8c8cb4095e656a611"
server_key_iv: "9d0607c13d0efa800e7522f4437c4d86"
server_key_version: "encryptionkey@0"
I20260923 18:00:35.898826 22501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:35.907788 22501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:35.910075 22501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:35.915452 22501 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.189:44001
I20260923 18:00:35.916143 22501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
I20260923 18:00:35.916592 22674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.189:44001 every 8 connection(s)
I20260923 18:00:35.917840 22675 sys_catalog.cc:263] Verifying existing consensus state
I20260923 18:00:35.918720 22675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: Bootstrap starting.
I20260923 18:00:35.927976 22675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 18:00:35.928344 22675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: Bootstrap complete.
I20260923 18:00:35.928933 22675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } }
I20260923 18:00:35.929091 22675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 460156afb37e44d5a0c97e63447c0c7c, State: Initialized, Role: LEARNER
I20260923 18:00:35.929219 22675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } }
I20260923 18:00:35.929608 22678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } }
I20260923 18:00:35.929790 22678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: This master's current role is: LEARNER
I20260923 18:00:35.929798 22675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:35.933039 22689 catalog_manager.cc:1259] Loaded cluster ID: c84c2c9cb9bd495391ee03b26f00e237
I20260923 18:00:35.933157 22689 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: loading cluster ID for follower catalog manager: success
I20260923 18:00:35.934998 22689 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: acquiring CA information for follower catalog manager: success
I20260923 18:00:35.936218 22689 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260923 18:00:36.024948 22605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:36.025281 22605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:36.025326 22605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:36.029455 22605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:36.029516 22605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:36.029536 22605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:36.029552 22605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:36.029568 22605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:36.036155 22605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:41269,127.15.139.189:44001,127.15.139.188:37763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.188.keytab
--principal=oryx/127.15.139.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.188:37763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.139.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:36.037278 22605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:36.038317 22605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:36.046732 22697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:36.046933 22605 server_base.cc:1034] running on GCE node
W20260923 18:00:36.046746 22695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:36.046759 22694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:36.047768 22605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:36.048960 22605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:36.050164 22605 hybrid_clock.cc:648] HybridClock initialized: now 1790186436050135 us; error 41 us; skew 500 ppm
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:36.053321 22605 init.cc:377] Logged in from keytab as oryx/127.15.139.188@KRBTEST.COM (short username oryx)
I20260923 18:00:36.054248 22605 webserver.cc:460] Webserver started at http://127.15.139.188:46303/ using document root <none> and password file <none>
I20260923 18:00:36.054785 22605 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:36.054843 22605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:36.055020 22605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:36.057209 22605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data/instance:
uuid: "647b766238a14c569a467052c41b4ec1"
format_stamp: "Formatted at 2026-09-23 18:00:36 on dist-test-slave-vr57"
server_key: "cb43c21690bfdf11cc1c1291bce99f04"
server_key_iv: "0c14e75f2e3a6f4be01d6dbb76bd1027"
server_key_version: "encryptionkey@0"
I20260923 18:00:36.057729 22605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/wal/instance:
uuid: "647b766238a14c569a467052c41b4ec1"
format_stamp: "Formatted at 2026-09-23 18:00:36 on dist-test-slave-vr57"
server_key: "cb43c21690bfdf11cc1c1291bce99f04"
server_key_iv: "0c14e75f2e3a6f4be01d6dbb76bd1027"
server_key_version: "encryptionkey@0"
I20260923 18:00:36.059742 22228 heartbeater.cc:503] Master 127.15.139.190:41269 was elected leader, sending a full tablet report...
I20260923 18:00:36.062547 22605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260923 18:00:36.065120 22704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:36.066577 22605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:00:36.066699 22605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/wal
uuid: "647b766238a14c569a467052c41b4ec1"
format_stamp: "Formatted at 2026-09-23 18:00:36 on dist-test-slave-vr57"
server_key: "cb43c21690bfdf11cc1c1291bce99f04"
server_key_iv: "0c14e75f2e3a6f4be01d6dbb76bd1027"
server_key_version: "encryptionkey@0"
I20260923 18:00:36.066851 22605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:36.082917 22605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:36.086231 22605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:36.086414 22605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:36.094480 22605 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.188:37763
I20260923 18:00:36.094795 22756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.188:37763 every 8 connection(s)
I20260923 18:00:36.095645 22605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
I20260923 18:00:36.097527 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 22605
I20260923 18:00:36.099045 22757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:36.103483 22757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.139.188@KRBTEST.COM
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.188@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:36.113894 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.099146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56304 (local address 127.15.139.188:37763)
0923 18:00:36.099649 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.099665 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.099702 (+ 37us) server_negotiation.cc:407] Connection header received
0923 18:00:36.100493 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.100512 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.100846 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.101129 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.102084 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.103139 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.104348 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.104846 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.105081 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.108182 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.108195 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.108207 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.108244 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.110260 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.110739 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.110751 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.110757 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.110910 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.111262 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.111265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.111266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.111806 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.112041 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.112380 (+ 339us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.112721 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":142,"threads_started":1}
I20260923 18:00:36.115692 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.105687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.15.139.190:41269)
0923 18:00:36.105827 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.105831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.106128 (+ 297us) server_negotiation.cc:407] Connection header received
0923 18:00:36.106336 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.106347 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.106397 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.106534 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.107410 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.108026 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.109389 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.109635 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.109717 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.112134 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.112137 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.112139 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.112163 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.113783 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.114303 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.114307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.114309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.114405 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.114697 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.114700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.114702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.114930 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.115038 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.115383 (+ 345us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.115565 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 18:00:36.115737 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.105464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42148)
0923 18:00:36.106000 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.106040 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.106189 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.106579 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.106585 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.107068 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.107261 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.107269 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.108199 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.108202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.109252 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.109268 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.109753 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.109757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.109867 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.110337 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.110354 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.111980 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.113901 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.113909 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.113913 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.114211 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.114519 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.114521 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.114523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.114604 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.115059 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.115065 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.115181 (+ 116us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.115325 (+ 144us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.115521 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":174,"threads_started":1}
I20260923 18:00:36.116847 22757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.188@KRBTEST.COM for oryx/127.15.139.189@KRBTEST.COM
I20260923 18:00:36.125142 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.117283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44001 (local address 127.0.0.1:33076)
0923 18:00:36.117398 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.117421 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.117481 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.117926 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.117928 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.118098 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.118204 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.118209 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.119147 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.119153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.119757 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.119765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.120227 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.120230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.120324 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.120765 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.120775 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.122060 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.123771 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.123773 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.123775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.123969 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.124213 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.124215 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.124217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.124293 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.124671 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.124674 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.124766 (+ 92us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.124875 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.125009 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260923 18:00:36.125182 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.117407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33076 (local address 127.15.139.189:44001)
0923 18:00:36.117718 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.117721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.117733 (+ 12us) server_negotiation.cc:407] Connection header received
0923 18:00:36.117770 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.117771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.117810 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.117914 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.118320 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.119025 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.119873 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.120118 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.120232 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.122188 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.122191 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.122193 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.122217 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.123676 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.124061 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.124063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.124064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.124135 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.124368 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.124370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.124371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.124552 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.124633 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.124923 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.125021 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":101,"threads_started":1}
I20260923 18:00:36.126046 22757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.188@KRBTEST.COM for oryx/127.15.139.188@KRBTEST.COM
I20260923 18:00:36.134513 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.126810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56314 (local address 127.15.139.188:37763)
0923 18:00:36.126921 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.126924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.126992 (+ 68us) server_negotiation.cc:407] Connection header received
0923 18:00:36.127060 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.127062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.127095 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.127159 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.127481 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.128027 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.129272 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.129508 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.129599 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.131573 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.131580 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.131585 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.131612 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.132968 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.133403 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.133409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.133411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.133481 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.133745 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.133748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.133750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.133912 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.134010 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.134249 (+ 239us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.134389 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":47}
I20260923 18:00:36.134513 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.126598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:37763 (local address 127.0.0.1:56314)
0923 18:00:36.126915 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.126927 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.126995 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.127189 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.127191 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.127314 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.127384 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.127388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.128626 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.128629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.129136 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.129144 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.129636 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.129638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.129701 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.130168 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.130179 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.131423 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.133119 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.133123 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.133125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.133310 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.133573 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.133576 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.133578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.133655 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.134010 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.134012 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.134095 (+ 83us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.134195 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.134334 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254}
I20260923 18:00:36.138075 22757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: Bootstrap starting.
I20260923 18:00:36.140552 22757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:36.141316 22757 log.cc:824] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:36.143455 22757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: No bootstrap required, opened a new log
I20260923 18:00:36.146906 22757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } }
I20260923 18:00:36.147214 22757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:36.147306 22757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647b766238a14c569a467052c41b4ec1, State: Initialized, Role: FOLLOWER
I20260923 18:00:36.147876 22757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } }
I20260923 18:00:36.148460 22763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } } }
I20260923 18:00:36.148593 22763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:00:36.149252 22757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: configured and running, proceeding with master startup.
W20260923 18:00:36.154398 22774 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:00:36.154503 22774 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:00:36.157998 22605 master_runner.cc:428] Detected that this master 647b766238a14c569a467052c41b4ec1 is joining an existing cluster
I20260923 18:00:36.158106 22605 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.139.188:37763
I20260923 18:00:36.167903 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 22099
I20260923 18:00:36.170389 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.160695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56328 (local address 127.15.139.188:37763)
0923 18:00:36.160796 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.160803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.161178 (+ 375us) server_negotiation.cc:407] Connection header received
0923 18:00:36.161342 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.161348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.161389 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.161441 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.162260 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.163027 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.164525 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.164864 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.164929 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.166302 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.166306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.166308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.166329 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.168321 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.168852 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.168858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.168863 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.168941 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.169238 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.169244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.169249 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.169497 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.169571 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.169993 (+ 422us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.170132 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":41}
I20260923 18:00:36.171304 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.160440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:37763 (local address 127.0.0.1:56328)
0923 18:00:36.161088 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.161105 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.161238 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.161570 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.161575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.161916 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.162137 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.162146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.163165 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.163171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.164409 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.164420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.165051 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.165054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.165200 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.165800 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.165811 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.166162 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.168481 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.168486 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.168492 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.168753 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.169041 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.169046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.169051 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.169148 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.169651 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.169656 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.169757 (+ 101us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.170808 (+ 1051us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.171014 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":125,"thread_start_us":128,"threads_started":1}
I20260923 18:00:36.176309 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:46199
--local_ip_for_outbound_sockets=127.15.139.129
--tserver_master_addrs=127.15.139.190:41269
--webserver_port=33561
--webserver_interface=127.15.139.129
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab
--principal=oryx/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.139.190:41269,127.15.139.189:44001
--tserver_master_addrs=127.15.139.190:41269,127.15.139.189:44001,127.15.139.188:37763 with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:00:36.189356 22782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.160888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42152)
0923 18:00:36.161252 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.161265 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.161378 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.161748 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.161753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.162025 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.162197 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.162207 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.163215 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.163221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.164761 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.164769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.165469 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.165473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.165592 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.184280 (+ 18688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.184307 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.185067 (+ 760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.187614 (+ 2547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.187620 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.187624 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.187879 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.188210 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.188213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.188215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.188298 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.188776 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.188779 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.188879 (+ 100us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.188975 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.189151 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":152,"thread_start_us":117,"threads_started":1}
I20260923 18:00:36.189747 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.160511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.15.139.189:44001)
0923 18:00:36.160635 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.160639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.161218 (+ 579us) server_negotiation.cc:407] Connection header received
0923 18:00:36.161384 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.161387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.161432 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.161495 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.162291 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.163116 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.164412 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.164782 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.164866 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.185326 (+ 20460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.185332 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.185336 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.185370 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.187775 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.188306 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.188310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.188312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.188410 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.188735 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.188738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.188740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.188970 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.189052 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.189430 (+ 378us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.189575 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 18:00:36.190004 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.160433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42152 (local address 127.15.139.190:41269)
0923 18:00:36.160624 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.160629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.161330 (+ 701us) server_negotiation.cc:407] Connection header received
0923 18:00:36.161479 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.161482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.161536 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.161602 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.162321 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.163092 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.164882 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.165220 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.165304 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.185328 (+ 20024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.185335 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.185340 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.185382 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.187422 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.188003 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.188007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.188009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.188109 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.188390 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.188393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.188394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.188641 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.188732 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.189718 (+ 986us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.189854 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260923 18:00:36.190450 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.160742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44001 (local address 127.0.0.1:33084)
0923 18:00:36.161137 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.161151 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.161281 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.161645 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.161648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.161980 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.162153 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.162161 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.163265 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.163269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.164264 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.164274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.165044 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.165047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.165177 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.184404 (+ 19227us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.184425 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.185064 (+ 639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.187923 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.187927 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.187929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.188181 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.188534 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.188537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.188539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.188627 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.189133 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.189136 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.189229 (+ 93us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.189959 (+ 730us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.190123 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":109,"spinlock_wait_cycles":896,"thread_start_us":162,"threads_started":1}
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186434, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.190@KRBTEST.COM for oryx/127.15.139.188@KRBTEST.COM
I20260923 18:00:36.202481 22763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:36.202687 22763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } }
I20260923 18:00:36.203867 22763 leader_election.cc:288] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269), 460156afb37e44d5a0c97e63447c0c7c (127.15.139.189:44001)
I20260923 18:00:36.204515 22649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "647b766238a14c569a467052c41b4ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460156afb37e44d5a0c97e63447c0c7c" is_pre_election: true
I20260923 18:00:36.204712 22649 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 LEARNER]: Handling vote request from an unknown peer 647b766238a14c569a467052c41b4ec1
I20260923 18:00:36.204857 22649 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 647b766238a14c569a467052c41b4ec1 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260923 18:00:36.205624 22047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "647b766238a14c569a467052c41b4ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be70c9d1f51f405c8c9e59aa78004870" is_pre_election: true
I20260923 18:00:36.206117 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.194542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.15.139.188:37763)
0923 18:00:36.194664 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.194669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.194689 (+ 20us) server_negotiation.cc:407] Connection header received
0923 18:00:36.194810 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.194813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.194872 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.194958 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.195646 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.196417 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.197796 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.198146 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.198268 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.200986 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.200990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.200995 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.201030 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.203174 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.203793 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.203799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.203801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.203894 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.204313 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.204317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.204319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.204577 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.205120 (+ 543us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.205735 (+ 615us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.205959 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 18:00:36.206143 22708 leader_election.cc:302] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647b766238a14c569a467052c41b4ec1; no voters: 460156afb37e44d5a0c97e63447c0c7c, be70c9d1f51f405c8c9e59aa78004870
I20260923 18:00:36.206359 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.194432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:37763 (local address 127.0.0.1:56344)
0923 18:00:36.194555 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.194573 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.194676 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.195010 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.195013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.195243 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.195368 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.195375 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.196561 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.196568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.197655 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.197667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.198281 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.198286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.198440 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.199001 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.199017 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.200802 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.203329 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.203333 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.203335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.203668 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.204106 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.204109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.204111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.204209 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.205162 (+ 953us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.205166 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.205280 (+ 114us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.205382 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.205558 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 18:00:36.207077 22763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:36.208568 22763 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 18:00:36.209707 22037 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.139.188:37763: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "647b766238a14c569a467052c41b4ec1"
member_type: NON_VOTER
last_known_addr {
host: "127.15.139.188"
port: 37763
}
attrs {
promote: true
}
}
dest_uuid: "be70c9d1f51f405c8c9e59aa78004870"
cas_config_opid_index: 5
I20260923 18:00:36.210515 22037 consensus_queue.cc:237] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } }
I20260923 18:00:36.211629 22598 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 647b766238a14c569a467052c41b4ec1 (127.15.139.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.212831 22731 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 FOLLOWER]: Refusing update from remote peer be70c9d1f51f405c8c9e59aa78004870: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 18:00:36.213397 22597 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:36.213533 22598 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.139.188:37763
I20260923 18:00:36.213876 22598 sys_catalog.cc:455] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.213989 22598 sys_catalog.cc:458] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:36.214555 22605 master.cc:537] Master@127.15.139.188:37763 shutting down...
I20260923 18:00:36.219044 22731 raft_consensus.cc:858] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } }
W20260923 18:00:36.219157 22731 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } }
I20260923 18:00:36.220783 22731 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 460156afb37e44d5a0c97e63447c0c7c (127.15.139.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } }
I20260923 18:00:36.221421 22785 mvcc.cc:205] Tried to move back new op lower bound from 7332603636955340800 to 7332603636774113280. Current Snapshot: MvccSnapshot[applied={T|T < 7332603636810203136}]
I20260923 18:00:36.224929 22763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be70c9d1f51f405c8c9e59aa78004870. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } }
I20260923 18:00:36.225050 22763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260923 18:00:36.225144 22763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } }
I20260923 18:00:36.225200 22763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260923 18:00:36.225255 22763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } }
I20260923 18:00:36.225322 22763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260923 18:00:36.226200 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.211709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44001 (local address 127.0.0.1:33096)
0923 18:00:36.212458 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.212473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.212635 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.213073 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.213079 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.213102 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:36.213218 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.213228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.222642 (+ 9414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.222646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.224180 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.224196 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.225694 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.225698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.225805 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.225865 (+ 60us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.225954 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.226047 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668}
I20260923 18:00:36.226214 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.211973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.15.139.189:44001)
0923 18:00:36.212117 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.212123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.212545 (+ 422us) server_negotiation.cc:407] Connection header received
0923 18:00:36.212754 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.212758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.212827 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.212899 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:36.213412 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.214179 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.224376 (+ 10197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.225536 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.225693 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.225849 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.225972 (+ 123us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.226059 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 18:00:36.227098 22649 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 LEARNER]: Refusing update from remote peer be70c9d1f51f405c8c9e59aa78004870: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 18:00:36.227679 22784 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260923 18:00:36.229940 22649 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 647b766238a14c569a467052c41b4ec1 (127.15.139.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.230898 22599 sys_catalog.cc:455] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.231063 22599 sys_catalog.cc:458] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:36.231145 22605 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260923 18:00:36.231441 22605 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260923 18:00:36.231561 22605 tablet_replica.cc:330] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: stopping tablet replica
W20260923 18:00:36.232378 22023 consensus_peers.cc:487] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 -> Peer 647b766238a14c569a467052c41b4ec1 (127.15.139.188:37763): Couldn't send request to peer 647b766238a14c569a467052c41b4ec1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 18:00:36.232705 22678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader be70c9d1f51f405c8c9e59aa78004870. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.232829 22678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: This master's current role is: LEARNER
I20260923 18:00:36.232966 22678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: NON_VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: true } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.233047 22678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: This master's current role is: LEARNER
I20260923 18:00:36.237457 22599 raft_consensus.cc:1050] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870: attempting to promote NON_VOTER 460156afb37e44d5a0c97e63447c0c7c to VOTER
I20260923 18:00:36.237985 22599 consensus_queue.cc:237] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } }
I20260923 18:00:36.238499 22605 master.cc:552] Master@127.15.139.188:37763 shutdown complete.
I20260923 18:00:36.238602 22605 master_runner.cc:305] Clearing existing system tablet
I20260923 18:00:36.238902 22605 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:36.239003 22605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:36.239862 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.239368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:37763 (local address 127.0.0.1:56354)
0923 18:00:36.239677 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.239762 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:37763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260923 18:00:36.240586 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.240350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:37763 (local address 127.0.0.1:56360)
0923 18:00:36.240471 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.240515 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:37763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260923 18:00:36.240703 22023 consensus_peers.cc:487] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 -> Peer 647b766238a14c569a467052c41b4ec1 (127.15.139.188:37763): Couldn't send request to peer 647b766238a14c569a467052c41b4ec1. Status: Network error: Client connection negotiation failed: client connection to 127.15.139.188:37763: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 18:00:36.241230 22605 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:36.241336 22649 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 LEARNER]: Refusing update from remote peer be70c9d1f51f405c8c9e59aa78004870: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260923 18:00:36.241884 22784 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260923 18:00:36.244112 22598 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 460156afb37e44d5a0c97e63447c0c7c (127.15.139.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.244728 22648 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 460156afb37e44d5a0c97e63447c0c7c (127.15.139.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.245616 22598 sys_catalog.cc:455] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.245715 22598 sys_catalog.cc:458] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:36.245831 22598 sys_catalog.cc:455] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.245893 22598 sys_catalog.cc:458] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:36.246601 22678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.246774 22678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:00:36.247682 22795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:36.247925 22605 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:36.248088 22605 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260923 18:00:36.248131 22605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/wal
uuid: "647b766238a14c569a467052c41b4ec1"
format_stamp: "Formatted at 2026-09-23 18:00:36 on dist-test-slave-vr57"
server_key: "cb43c21690bfdf11cc1c1291bce99f04"
server_key_iv: "0c14e75f2e3a6f4be01d6dbb76bd1027"
server_key_version: "encryptionkey@0"
I20260923 18:00:36.250540 22605 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:36.253237 22605 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 18:00:36.253345 22605 log.cc:1196] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260923 18:00:36.253664 22605 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: Deleting consensus metadata
I20260923 18:00:36.253836 22605 master_runner.cc:315] Copying system tablet from 127.15.139.190:41269
I20260923 18:00:36.255064 22605 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.139.190:41269
I20260923 18:00:36.265400 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.256082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42168)
0923 18:00:36.256350 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.256368 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.256497 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.256853 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.256856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.257077 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.257184 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.257191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.258387 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.258391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.259424 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.259435 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.260271 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.260275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.260362 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.261023 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.261039 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.261621 (+ 582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.263701 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.263705 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.263708 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.263966 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.264271 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.264275 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.264277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.264373 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.264826 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.264829 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.264936 (+ 107us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.265048 (+ 112us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.265202 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":112,"threads_started":1}
I20260923 18:00:36.265412 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.255839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42168 (local address 127.15.139.190:41269)
0923 18:00:36.255975 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.255980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.256588 (+ 608us) server_negotiation.cc:407] Connection header received
0923 18:00:36.256651 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.256654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.256732 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.256820 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.257321 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.258266 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.259596 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.260109 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.260274 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.261767 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.261771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.261774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.261802 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.263584 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.264073 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.264076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.264078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.264170 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.264459 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.264462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.264474 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.264719 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.264822 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.265117 (+ 295us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.265242 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 18:00:36.266301 22057 tablet_copy_service.cc:140] P be70c9d1f51f405c8c9e59aa78004870: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 647b766238a14c569a467052c41b4ec1 ({username='oryx', principal='oryx/127.15.139.188@KRBTEST.COM'} at 127.0.0.1:42168)
I20260923 18:00:36.266412 22057 tablet_copy_service.cc:161] P be70c9d1f51f405c8c9e59aa78004870: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 647b766238a14c569a467052c41b4ec1 at {username='oryx', principal='oryx/127.15.139.188@KRBTEST.COM'} at 127.0.0.1:42168: session id = 647b766238a14c569a467052c41b4ec1-00000000000000000000000000000000
I20260923 18:00:36.267598 22057 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 18:00:36.268963 22605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 18:00:36.271404 22605 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: tablet copy: Starting download of 0 data blocks...
I20260923 18:00:36.271651 22605 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: tablet copy: Starting download of 1 WAL segments...
I20260923 18:00:36.273241 22605 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 18:00:36.275166 22057 tablet_copy_service.cc:342] P be70c9d1f51f405c8c9e59aa78004870: Request end of tablet copy session 647b766238a14c569a467052c41b4ec1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.139.188@KRBTEST.COM'} at 127.0.0.1:42168
I20260923 18:00:36.275264 22057 tablet_copy_service.cc:433] P be70c9d1f51f405c8c9e59aa78004870: ending tablet copy session 647b766238a14c569a467052c41b4ec1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 647b766238a14c569a467052c41b4ec1
I20260923 18:00:36.283721 22605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:36.285672 22802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:36.288872 22605 server_base.cc:1034] running on GCE node
W20260923 18:00:36.288853 22805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:36.291723 22803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:36.292039 22605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:36.292402 22605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:36.293561 22605 hybrid_clock.cc:648] HybridClock initialized: now 1790186436293542 us; error 47 us; skew 500 ppm
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:36.295317 22605 init.cc:377] Logged in from keytab as oryx/127.15.139.188@KRBTEST.COM (short username oryx)
I20260923 18:00:36.295857 22605 webserver.cc:460] Webserver started at http://127.15.139.188:46031/ using document root <none> and password file <none>
I20260923 18:00:36.296063 22605 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:36.296108 22605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:36.297572 22605 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:36.299721 22812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:36.300019 22605 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:36.300101 22605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/wal
uuid: "647b766238a14c569a467052c41b4ec1"
format_stamp: "Formatted at 2026-09-23 18:00:36 on dist-test-slave-vr57"
server_key: "cb43c21690bfdf11cc1c1291bce99f04"
server_key_iv: "0c14e75f2e3a6f4be01d6dbb76bd1027"
server_key_version: "encryptionkey@0"
I20260923 18:00:36.300199 22605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:36.306401 22362 heartbeater.cc:503] Master 127.15.139.190:41269 was elected leader, sending a full tablet report...
I20260923 18:00:36.311851 22605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:36.314208 22605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:36.319543 22605 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.188:37763
I20260923 18:00:36.319541 22864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.188:37763 every 8 connection(s)
I20260923 18:00:36.320372 22605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
I20260923 18:00:36.322335 22865 sys_catalog.cc:263] Verifying existing consensus state
I20260923 18:00:36.323385 22865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: Bootstrap starting.
I20260923 18:00:36.331929 22865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 18:00:36.332231 22865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: Bootstrap complete.
I20260923 18:00:36.332868 22865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } }
I20260923 18:00:36.333009 22865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 647b766238a14c569a467052c41b4ec1, State: Initialized, Role: LEARNER
I20260923 18:00:36.333122 22865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } }
I20260923 18:00:36.333506 22868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.333604 22865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:36.333698 22868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: This master's current role is: LEARNER
I20260923 18:00:36.338143 22879 catalog_manager.cc:1259] Loaded cluster ID: c84c2c9cb9bd495391ee03b26f00e237
I20260923 18:00:36.338270 22879 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: loading cluster ID for follower catalog manager: success
I20260923 18:00:36.340554 22879 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: acquiring CA information for follower catalog manager: success
I20260923 18:00:36.342089 22879 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260923 18:00:36.346866 22783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:36.347162 22783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:36.347267 22783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:36.351599 22783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:36.351680 22783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:36.351788 22783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:36.357403 22783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab
--principal=oryx/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:46199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=33561
--tserver_master_addrs=127.15.139.190:41269,127.15.139.189:44001,127.15.139.188:37763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:36.358526 22783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:36.359726 22783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:36.369560 22885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:36.369555 22887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:36.369555 22884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:36.369727 22783 server_base.cc:1034] running on GCE node
I20260923 18:00:36.370340 22783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:36.371006 22783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:36.372269 22783 hybrid_clock.cc:648] HybridClock initialized: now 1790186436372229 us; error 47 us; skew 500 ppm
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:36.375129 22783 init.cc:377] Logged in from keytab as oryx/127.15.139.129@KRBTEST.COM (short username oryx)
I20260923 18:00:36.376065 22783 webserver.cc:460] Webserver started at http://127.15.139.129:33561/ using document root <none> and password file <none>
I20260923 18:00:36.376641 22783 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:36.376741 22783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:36.380621 22783 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:00:36.382740 22894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:36.384162 22783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:36.384265 22783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "e53e7fbff400481f8979b8602fcd6a9b"
format_stamp: "Formatted at 2026-09-23 18:00:34 on dist-test-slave-vr57"
server_key: "2a00457f1d0962b6423668c546e0e090"
server_key_iv: "478e68ffff060bec1ccbc811f7981866"
server_key_version: "encryptionkey@0"
I20260923 18:00:36.384799 22783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:36.399935 22783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:36.403429 22783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:36.403659 22783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:36.404227 22783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:36.405045 22783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:36.405092 22783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:36.405130 22783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:36.405146 22783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:36.414808 22783 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:46199
I20260923 18:00:36.414876 23007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:46199 every 8 connection(s)
I20260923 18:00:36.415956 22783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:36.426424 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 22783
I20260923 18:00:36.426657 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 22233
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.189@KRBTEST.COM
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.188@KRBTEST.COM
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:36.434199 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.417832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:60457 (local address 127.15.139.188:37763)
0923 18:00:36.418101 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.418106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.419610 (+ 1504us) server_negotiation.cc:407] Connection header received
0923 18:00:36.420690 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.420693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.420743 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.420847 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.422319 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.423422 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.424365 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.424662 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.424760 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.429155 (+ 4395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.429158 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.429161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.429189 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.431463 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.432226 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.432230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.432232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.432321 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.432733 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.432739 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.432744 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.432983 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.433058 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.433887 (+ 829us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.434024 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260923 18:00:36.434198 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.418532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:48995 (local address 127.15.139.189:44001)
0923 18:00:36.418665 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.418669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.419568 (+ 899us) server_negotiation.cc:407] Connection header received
0923 18:00:36.420982 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.420985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.421037 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.421142 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.422340 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.423001 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.424624 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.424928 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.425006 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.429154 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.429158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.429161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.429189 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.431463 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.432372 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.432376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.432378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.432477 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.432768 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.432774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.432778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.432971 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.433058 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.433887 (+ 829us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.434016 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 18:00:36.434732 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:36899
--local_ip_for_outbound_sockets=127.15.139.130
--tserver_master_addrs=127.15.139.190:41269
--webserver_port=39691
--webserver_interface=127.15.139.130
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab
--principal=oryx/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.139.190:41269,127.15.139.189:44001
--tserver_master_addrs=127.15.139.190:41269,127.15.139.189:44001,127.15.139.188:37763 with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:00:36.435984 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.417987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:34151 (local address 127.15.139.190:41269)
0923 18:00:36.418111 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.418116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.419668 (+ 1552us) server_negotiation.cc:407] Connection header received
0923 18:00:36.420570 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.420573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.420646 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.420748 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.422506 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.423338 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.424610 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.424925 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.425036 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.430959 (+ 5923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.430965 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.430971 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.431007 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.433008 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.433611 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.433617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.433622 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.433733 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.434096 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.434101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.434113 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.434363 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.434448 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.435591 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.435751 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 18:00:36.436020 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.418309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:37763 (local address 127.15.139.129:60457)
0923 18:00:36.419386 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.419432 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.420345 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.420931 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.420939 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.421520 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.422172 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.422187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.423542 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.423553 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.424235 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.424244 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.424811 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.424814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.425019 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.425772 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.425788 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.428948 (+ 3160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.431659 (+ 2711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.431666 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.431686 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.432088 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.432432 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.432435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.432437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.432622 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.433131 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.433138 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.433575 (+ 437us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.434438 (+ 863us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.434912 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"mutex_wait_us":358,"spinlock_wait_cycles":26496,"thread_start_us":141,"threads_started":1}
I20260923 18:00:36.436298 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.418309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44001 (local address 127.15.139.129:48995)
0923 18:00:36.419386 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.419432 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.420345 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.421175 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.421178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.421544 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.422174 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.422189 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.423154 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.423156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.424482 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.424493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.425172 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.425177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.425274 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.425976 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.425990 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.428942 (+ 2952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.431659 (+ 2717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.431666 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.431685 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.432260 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.432586 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.432588 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.432590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.432671 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.433131 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.433138 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.433575 (+ 437us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.434358 (+ 783us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.434845 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"mutex_wait_us":361,"spinlock_wait_cycles":121856,"thread_start_us":635,"threads_started":1}
I20260923 18:00:36.437229 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.418308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.129:34151)
0923 18:00:36.419385 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.419431 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.420327 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.420930 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.420943 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.421521 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.422174 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.422189 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.423459 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.423463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.424482 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.424493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.425059 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.425064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.425162 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.425836 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.425855 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.430782 (+ 4927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.433173 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.433179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.433184 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.433442 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.433886 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.433889 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.433890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.433981 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.435225 (+ 1244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.435231 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.435355 (+ 124us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.436898 (+ 1543us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.437060 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":180,"spinlock_wait_cycles":62336,"thread_start_us":167,"threads_started":1}
I20260923 18:00:36.437316 23008 heartbeater.cc:347] Connected to a master server at 127.15.139.188:37763
I20260923 18:00:36.437729 23008 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:36.438139 23009 heartbeater.cc:347] Connected to a master server at 127.15.139.190:41269
I20260923 18:00:36.438217 23009 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:36.437384 23010 heartbeater.cc:347] Connected to a master server at 127.15.139.189:44001
I20260923 18:00:36.438350 23009 heartbeater.cc:511] Master 127.15.139.190:41269 requested a full tablet report, sending...
I20260923 18:00:36.438408 23010 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:36.438350 23008 heartbeater.cc:511] Master 127.15.139.188:37763 requested a full tablet report, sending...
I20260923 18:00:36.438596 23010 heartbeater.cc:511] Master 127.15.139.189:44001 requested a full tablet report, sending...
I20260923 18:00:36.439640 22037 ts_manager.cc:194] Re-registered known tserver with Master: e53e7fbff400481f8979b8602fcd6a9b (127.15.139.129:46199)
I20260923 18:00:36.439956 22639 ts_manager.cc:194] Registered new tserver with Master: e53e7fbff400481f8979b8602fcd6a9b (127.15.139.129:46199)
I20260923 18:00:36.440415 22829 ts_manager.cc:194] Registered new tserver with Master: e53e7fbff400481f8979b8602fcd6a9b (127.15.139.129:46199)
I20260923 18:00:36.440490 22037 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:34151
I20260923 18:00:36.532280 22496 heartbeater.cc:503] Master 127.15.139.190:41269 was elected leader, sending a full tablet report...
W20260923 18:00:36.553493 23016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:36.553756 23016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:36.553799 23016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:36.557821 23016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:36.557884 23016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:36.557942 23016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:36.563611 23016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab
--principal=oryx/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:36899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=39691
--tserver_master_addrs=127.15.139.190:41269,127.15.139.189:44001,127.15.139.188:37763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:36.564610 23016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:36.565744 23016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:36.575383 23024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:36.575412 23021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:36.575431 23022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:36.575831 23016 server_base.cc:1034] running on GCE node
I20260923 18:00:36.576310 23016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:36.577001 23016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:36.578184 23016 hybrid_clock.cc:648] HybridClock initialized: now 1790186436578170 us; error 29 us; skew 500 ppm
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:36.580981 23016 init.cc:377] Logged in from keytab as oryx/127.15.139.130@KRBTEST.COM (short username oryx)
I20260923 18:00:36.581908 23016 webserver.cc:460] Webserver started at http://127.15.139.130:39691/ using document root <none> and password file <none>
I20260923 18:00:36.582540 23016 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:36.582677 23016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:36.586597 23016 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:36.588702 23031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:36.589991 23016 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:36.590147 23016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "b844678d1ab24411857fc79ae7010d2c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "6c496c5e6119a4968ace6483ee55a6fc"
server_key_iv: "7c03ea1e6f5b8902f5b88aa2cc4dee8d"
server_key_version: "encryptionkey@0"
I20260923 18:00:36.590705 23016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:36.616554 23016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:36.620025 23016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:36.620240 23016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:36.620870 23016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:36.621805 23016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:36.621879 23016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:36.621961 23016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:36.622011 23016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:36.632412 23016 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:36899
I20260923 18:00:36.632436 23144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:36899 every 8 connection(s)
I20260923 18:00:36.633638 23016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:36.643460 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 23016
I20260923 18:00:36.643708 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 22367
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.189@KRBTEST.COM
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:36.650733 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.635736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:46321 (local address 127.15.139.189:44001)
0923 18:00:36.635902 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.636026 (+ 124us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.637457 (+ 1431us) server_negotiation.cc:407] Connection header received
0923 18:00:36.638543 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.638546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.638606 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.638669 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.640254 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.641103 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.642577 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.642812 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.642937 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.645982 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.645986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.645989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.646023 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.647954 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.648681 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.648685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.648687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.648787 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.649275 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.649282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.649286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.649513 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.649604 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.650404 (+ 800us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.650533 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 18:00:36.650797 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.635631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:37195 (local address 127.15.139.190:41269)
0923 18:00:36.635796 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.635800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.637195 (+ 1395us) server_negotiation.cc:407] Connection header received
0923 18:00:36.638568 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.638571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.638631 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.638735 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.640238 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.640833 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.641984 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.642240 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.642349 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.646391 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.646395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.646397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.646440 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.648347 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.648923 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.648930 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.648935 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.649081 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.649406 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.649412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.649416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.649667 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.649755 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.650396 (+ 641us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.650541 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.188@KRBTEST.COM
I20260923 18:00:36.652554 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.636260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44001 (local address 127.15.139.130:46321)
0923 18:00:36.637335 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.637349 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.638296 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.638914 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.638921 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.639468 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.640095 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.640104 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.641254 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.641257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.642435 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.642442 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.642930 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.642934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.643058 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.643626 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.643641 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.645791 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.648104 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.648111 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.648132 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.648554 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.648909 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.648913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.648916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.649156 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.649682 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.649690 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.650098 (+ 408us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.650910 (+ 812us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.651358 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":995,"mutex_wait_us":402,"spinlock_wait_cycles":86912,"thread_start_us":132,"threads_started":1}
I20260923 18:00:36.652554 23149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.636260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.130:37195)
0923 18:00:36.637042 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.637096 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.638313 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.638914 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.638923 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.639451 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.640085 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.640099 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.641005 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.641007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.641864 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.641872 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.642396 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.642400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.642624 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.643438 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.643462 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.646258 (+ 2796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.648485 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.648491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.648496 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.648789 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.649199 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.649204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.649209 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.649304 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.649816 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.649822 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.650097 (+ 275us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.650942 (+ 845us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.651457 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":255,"thread_start_us":156,"threads_started":1}
I20260923 18:00:36.652753 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:45875
--local_ip_for_outbound_sockets=127.15.139.131
--tserver_master_addrs=127.15.139.190:41269
--webserver_port=39279
--webserver_interface=127.15.139.131
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab
--principal=oryx/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.139.190:41269,127.15.139.189:44001
--tserver_master_addrs=127.15.139.190:41269,127.15.139.189:44001,127.15.139.188:37763 with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:00:36.653821 23146 heartbeater.cc:347] Connected to a master server at 127.15.139.190:41269
I20260923 18:00:36.654106 23146 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:36.654790 23146 heartbeater.cc:511] Master 127.15.139.190:41269 requested a full tablet report, sending...
I20260923 18:00:36.654863 23147 heartbeater.cc:347] Connected to a master server at 127.15.139.189:44001
I20260923 18:00:36.654961 23147 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:36.655165 23147 heartbeater.cc:511] Master 127.15.139.189:44001 requested a full tablet report, sending...
I20260923 18:00:36.656062 22037 ts_manager.cc:194] Re-registered known tserver with Master: b844678d1ab24411857fc79ae7010d2c (127.15.139.130:36899)
I20260923 18:00:36.656497 23151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.636572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:37763 (local address 127.15.139.130:47743)
0923 18:00:36.637379 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.637393 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.638296 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.639051 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.639054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.639464 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.640096 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.640104 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.641254 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.641257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.642431 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.642442 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.648509 (+ 6067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.648514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.648658 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.649156 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.649170 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.651947 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.654716 (+ 2769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.654720 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.654722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.654970 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.655331 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.655334 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.655335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.655430 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.655979 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.655981 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.656067 (+ 86us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.656172 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.656321 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":196,"spinlock_wait_cycles":19328,"thread_start_us":101,"threads_started":1}
I20260923 18:00:36.656616 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.636671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:47743 (local address 127.15.139.188:37763)
0923 18:00:36.636795 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.636799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.637478 (+ 679us) server_negotiation.cc:407] Connection header received
0923 18:00:36.638864 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.638868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.638926 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.639011 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.640261 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.641103 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.642552 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.642924 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.643033 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.652129 (+ 9096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.652134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.652137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.652168 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.654580 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.655096 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.655099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.655108 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.655215 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.655574 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.655581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.655590 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.655832 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.655910 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.656249 (+ 339us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.656385 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:00:36.656908 22037 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:37195
I20260923 18:00:36.657295 23145 heartbeater.cc:347] Connected to a master server at 127.15.139.188:37763
I20260923 18:00:36.657368 23145 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:36.657547 23145 heartbeater.cc:511] Master 127.15.139.188:37763 requested a full tablet report, sending...
I20260923 18:00:36.657953 22639 ts_manager.cc:194] Registered new tserver with Master: b844678d1ab24411857fc79ae7010d2c (127.15.139.130:36899)
I20260923 18:00:36.658214 22829 ts_manager.cc:194] Registered new tserver with Master: b844678d1ab24411857fc79ae7010d2c (127.15.139.130:36899)
I20260923 18:00:36.667340 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.663780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:37763 (local address 127.0.0.1:55922)
0923 18:00:36.663920 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.663939 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.664047 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.664311 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.664317 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.664342 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:36.664468 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.664476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.665285 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.665288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.666063 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.666072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.667013 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.667016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.667090 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.667111 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.667171 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.667231 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 18:00:36.667373 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.663857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.15.139.188:37763)
0923 18:00:36.664055 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.664059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.664074 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:00:36.664142 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.664144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.664193 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.664270 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:36.664590 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.665124 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.666191 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.666875 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.667008 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.667154 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.667191 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.667254 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260923 18:00:36.668129 22839 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260923 18:00:36.668427 22868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be70c9d1f51f405c8c9e59aa78004870. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.668602 22868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: This master's current role is: LEARNER
W20260923 18:00:36.772583 23152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:36.772848 23152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:36.772893 23152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:36.777083 23152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:36.777143 23152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:36.777235 23152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:36.782610 23152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab
--principal=oryx/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:45875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=39279
--tserver_master_addrs=127.15.139.190:41269,127.15.139.189:44001,127.15.139.188:37763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:36.783732 23152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:36.784749 23152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:36.794536 23157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:36.794556 23160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:36.794667 23152 server_base.cc:1034] running on GCE node
W20260923 18:00:36.794536 23158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:36.795372 23152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:36.796056 23152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:36.797222 23152 hybrid_clock.cc:648] HybridClock initialized: now 1790186436797203 us; error 30 us; skew 500 ppm
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:36.800385 23152 init.cc:377] Logged in from keytab as oryx/127.15.139.131@KRBTEST.COM (short username oryx)
I20260923 18:00:36.801306 23152 webserver.cc:460] Webserver started at http://127.15.139.131:39279/ using document root <none> and password file <none>
I20260923 18:00:36.801894 23152 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:36.802021 23152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:36.806046 23152 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:00:36.808336 23167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:36.809669 23152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:36.809818 23152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "f248c4fde5a24ad28b187849e4eb225c"
format_stamp: "Formatted at 2026-09-23 18:00:35 on dist-test-slave-vr57"
server_key: "033aa4ee648887e55b742ed4de2f5e05"
server_key_iv: "d716c7cb9f1848feb7662ba0c1f1410a"
server_key_version: "encryptionkey@0"
I20260923 18:00:36.810402 23152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:36.838440 23152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:36.841647 23152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:36.841854 23152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:36.842458 23152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:36.843351 23152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:36.843422 23152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:36.843525 23152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:36.843564 23152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:36.852828 23152 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:45875
I20260923 18:00:36.852871 23280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:45875 every 8 connection(s)
I20260923 18:00:36.853852 23152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:36.861246 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 23152
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.189@KRBTEST.COM
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.188@KRBTEST.COM
Sep 23 18:00:36 dist-test-slave-vr57 krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186436, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:36.873793 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.856169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:59137 (local address 127.15.139.188:37763)
0923 18:00:36.856282 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.856287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.857094 (+ 807us) server_negotiation.cc:407] Connection header received
0923 18:00:36.858244 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.858248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.858304 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.858355 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.859785 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.860609 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.862367 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.862670 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.862750 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.868509 (+ 5759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.868513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.868515 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.868543 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.872009 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.872455 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.872458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.872460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.872544 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.872812 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.872815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.872817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.873030 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.873116 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.873492 (+ 376us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.873619 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 18:00:36.874614 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.856382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.131:49723)
0923 18:00:36.857417 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.857429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.858010 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.858575 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.858582 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.858980 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.859563 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.859587 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.860696 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.860698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.862272 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.862282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.862957 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.862961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.863316 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.864146 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.864168 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.867699 (+ 3531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.871271 (+ 3572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.871277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.871282 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.871567 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.871902 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.871907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.871914 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.872056 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.872596 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.872599 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.872868 (+ 269us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.873086 (+ 218us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.873463 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":286,"spinlock_wait_cycles":8064,"thread_start_us":193,"threads_started":1}
I20260923 18:00:36.875826 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.856382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:37763 (local address 127.15.139.131:59137)
0923 18:00:36.856927 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.856980 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.858014 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.858577 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.858584 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.859037 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.859571 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.859587 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.860762 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.860769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.862206 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.862221 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.862957 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.862961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.863336 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.864238 (+ 902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.864254 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.868354 (+ 4100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.872096 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.872100 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.872105 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.872339 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.872641 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.872644 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.872646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.872724 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.873157 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.873160 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.873244 (+ 84us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.873347 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.875276 (+ 1929us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":2123,"spinlock_wait_cycles":32128,"thread_start_us":161,"threads_started":1}
I20260923 18:00:36.875988 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.863481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41534 (local address 127.15.139.190:41269)
0923 18:00:36.865862 (+ 2381us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.865869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.865882 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:00:36.865917 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.865920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.865971 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.866054 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.867064 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.867896 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.869231 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.869630 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.870121 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.871383 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.871386 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.871389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.871411 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.873177 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.873646 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.873649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.873651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.873730 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.874369 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.874372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.874374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.874821 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.875565 (+ 744us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.875643 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.875790 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2302,"thread_start_us":83,"threads_started":1}
I20260923 18:00:36.876490 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.855789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:49723 (local address 127.15.139.190:41269)
0923 18:00:36.855952 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.855959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.857528 (+ 1569us) server_negotiation.cc:407] Connection header received
0923 18:00:36.858233 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.858236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.858298 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.858403 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.859741 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.860582 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.862393 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.862737 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.862803 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.867897 (+ 5094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.867902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.867905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.867935 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.871141 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.871692 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.871696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.871699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.871789 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.872183 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.872188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.872190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.872433 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.872532 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.876233 (+ 3701us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.876360 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 18:00:36.876811 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.856402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:48181 (local address 127.15.139.189:44001)
0923 18:00:36.856533 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.856539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.857699 (+ 1160us) server_negotiation.cc:407] Connection header received
0923 18:00:36.858243 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.858247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.858300 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.858355 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.859776 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.860346 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.862336 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.862668 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.862745 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.867986 (+ 5241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.867990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.867992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.868015 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.870180 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.870810 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.870814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.870815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.870900 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.871297 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.871300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.871302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.871568 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.871665 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.876502 (+ 4837us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.876630 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":44}
I20260923 18:00:36.877621 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.856382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44001 (local address 127.15.139.131:48181)
0923 18:00:36.857618 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.857627 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.858014 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.858577 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.858584 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.859037 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:36.859568 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.859588 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.860551 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.860554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.862192 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.862203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.862957 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.862961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.863432 (+ 471us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.864161 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:36.864174 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:36.867852 (+ 3678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:36.870298 (+ 2446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.870304 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.870321 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.870696 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.871015 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:36.871019 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:36.871023 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:36.871195 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:36.871896 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:36.871903 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:36.874593 (+ 2690us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.877271 (+ 2678us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.877423 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1170,"mutex_wait_us":608,"spinlock_wait_cycles":1280,"thread_start_us":101,"threads_started":1}
I20260923 18:00:36.877624 23282 heartbeater.cc:347] Connected to a master server at 127.15.139.190:41269
I20260923 18:00:36.878135 23282 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:36.878454 23283 heartbeater.cc:347] Connected to a master server at 127.15.139.189:44001
I20260923 18:00:36.878548 23283 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:36.878741 23282 heartbeater.cc:511] Master 127.15.139.190:41269 requested a full tablet report, sending...
I20260923 18:00:36.878741 23283 heartbeater.cc:511] Master 127.15.139.189:44001 requested a full tablet report, sending...
I20260923 18:00:36.877624 23281 heartbeater.cc:347] Connected to a master server at 127.15.139.188:37763
I20260923 18:00:36.879287 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.864813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55936 (local address 127.15.139.188:37763)
0923 18:00:36.866268 (+ 1455us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.866275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.866287 (+ 12us) server_negotiation.cc:407] Connection header received
0923 18:00:36.866326 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.866331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.866381 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.867249 (+ 868us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.867340 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.868103 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.873095 (+ 4992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.873437 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.873964 (+ 527us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.875157 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.875161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.875166 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.875194 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.877034 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.877534 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.877537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.877539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.877616 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.877945 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.877950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.877953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.878183 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.878899 (+ 716us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.878971 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.879088 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1378,"thread_start_us":97,"threads_started":1}
I20260923 18:00:36.879328 23281 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:36.879488 23281 heartbeater.cc:511] Master 127.15.139.188:37763 requested a full tablet report, sending...
I20260923 18:00:36.879918 22639 ts_manager.cc:194] Registered new tserver with Master: f248c4fde5a24ad28b187849e4eb225c (127.15.139.131:45875)
I20260923 18:00:36.880177 22828 ts_manager.cc:194] Registered new tserver with Master: f248c4fde5a24ad28b187849e4eb225c (127.15.139.131:45875)
I20260923 18:00:36.880565 22037 ts_manager.cc:194] Re-registered known tserver with Master: f248c4fde5a24ad28b187849e4eb225c (127.15.139.131:45875)
I20260923 18:00:36.881312 22037 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:49723
I20260923 18:00:36.882745 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.865676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.15.139.189:44001)
0923 18:00:36.866022 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.866035 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.866048 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:00:36.866088 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.866091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.866144 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.866239 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:36.869027 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.869873 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.871081 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.871691 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.871776 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.878245 (+ 6469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:36.878251 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:36.878255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:36.878283 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:36.880770 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.881247 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.881250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.881252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.881336 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:36.881885 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:36.881888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:36.881889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:36.882067 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:36.882216 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.882449 (+ 233us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.882573 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":77,"threads_started":1}
I20260923 18:00:36.883939 22037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:00:36.886991 22037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:36.890017 23299 sys_catalog.cc:455] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: NON_VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: true } } }
I20260923 18:00:36.890182 23299 sys_catalog.cc:458] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:36.893080 22784 raft_consensus.cc:1050] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870: attempting to promote NON_VOTER 647b766238a14c569a467052c41b4ec1 to VOTER
I20260923 18:00:36.893522 22784 consensus_queue.cc:237] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } }
I20260923 18:00:36.894809 22839 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 LEARNER]: Refusing update from remote peer be70c9d1f51f405c8c9e59aa78004870: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260923 18:00:36.894934 22648 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 FOLLOWER]: Refusing update from remote peer be70c9d1f51f405c8c9e59aa78004870: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260923 18:00:36.895249 23305 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260923 18:00:36.895431 22784 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260923 18:00:36.897226 22784 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 647b766238a14c569a467052c41b4ec1 (127.15.139.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } } }
I20260923 18:00:36.897857 22839 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 647b766238a14c569a467052c41b4ec1 (127.15.139.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } } }
I20260923 18:00:36.898991 22784 sys_catalog.cc:455] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } } }
I20260923 18:00:36.899098 22784 sys_catalog.cc:458] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:36.899072 23305 sys_catalog.cc:455] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } } }
I20260923 18:00:36.899200 23305 sys_catalog.cc:458] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:36.899792 22868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } } }
I20260923 18:00:36.899959 22868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:00:36.900672 23299 sys_catalog.cc:455] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } } }
I20260923 18:00:36.900790 23299 sys_catalog.cc:458] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:36.901744 22648 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 647b766238a14c569a467052c41b4ec1 (127.15.139.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } } }
I20260923 18:00:36.904407 23309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be70c9d1f51f405c8c9e59aa78004870" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } } }
I20260923 18:00:36.905763 23309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:00:36.911024 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.904649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:36899 (local address 127.0.0.1:59720)
0923 18:00:36.905652 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.905669 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.905769 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.906703 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.906706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.906720 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:36.906830 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.906836 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.908317 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.908323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.909536 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.909545 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.910563 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.910566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.910666 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.910694 (+ 28us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.910778 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.910854 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"mutex_wait_us":102,"thread_start_us":93,"threads_started":1}
I20260923 18:00:36.911078 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.904928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59720 (local address 127.15.139.130:36899)
0923 18:00:36.906195 (+ 1267us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.906210 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.906235 (+ 25us) server_negotiation.cc:407] Connection header received
0923 18:00:36.906316 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.906320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.906537 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.906718 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:36.906980 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.908156 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.909660 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.910434 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.910567 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.910703 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.910806 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.910926 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":114,"threads_started":1}
I20260923 18:00:36.911312 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.904310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:46199 (local address 127.0.0.1:53242)
0923 18:00:36.904478 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.904499 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.904605 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.906542 (+ 1937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.906546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.906566 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:36.906717 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.906725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.908344 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.908347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.909843 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.909854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.910941 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.910945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.911035 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.911059 (+ 24us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.911112 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.911187 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":127}
I20260923 18:00:36.911913 23313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.904967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:45875 (local address 127.0.0.1:56234)
0923 18:00:36.905983 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:36.905999 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:36.906092 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:36.907261 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:36.907264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:36.907284 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:36.907375 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.907382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.909100 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.909104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.910284 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:36.910292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.911583 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.911587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:36.911692 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.911713 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 18:00:36.911756 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 18:00:36.911810 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":943,"mutex_wait_us":33,"thread_start_us":92,"threads_started":1}
I20260923 18:00:36.912420 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.904580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53242 (local address 127.15.139.129:46199)
0923 18:00:36.906067 (+ 1487us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.906082 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.906110 (+ 28us) server_negotiation.cc:407] Connection header received
0923 18:00:36.906197 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.906202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.906384 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.906526 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:36.906843 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.908185 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.909977 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.910792 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.911982 (+ 1190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.912118 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.912186 (+ 68us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.912285 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":73,"threads_started":1}
I20260923 18:00:36.912935 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:36.905263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56234 (local address 127.15.139.131:45875)
0923 18:00:36.905572 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:36.905585 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:36.906897 (+ 1312us) server_negotiation.cc:407] Connection header received
0923 18:00:36.906973 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:36.906977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:36.907115 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:36.907282 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:36.907483 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.908956 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.910413 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:36.911400 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:36.912283 (+ 883us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:36.912704 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:36.912748 (+ 44us) server_negotiation.cc:299] Negotiation successful
0923 18:00:36.912825 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":130,"threads_started":1}
I20260923 18:00:36.914506 23079 tablet_service.cc:1467] Processing CreateTablet for tablet 18990f30935f46ceb7c68b370f68b82d (DEFAULT_TABLE table=test-table [id=31bbab7fb5c64f3b998c107a7187722f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:36.915671 23079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18990f30935f46ceb7c68b370f68b82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:36.915963 23215 tablet_service.cc:1467] Processing CreateTablet for tablet 18990f30935f46ceb7c68b370f68b82d (DEFAULT_TABLE table=test-table [id=31bbab7fb5c64f3b998c107a7187722f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:36.916960 23215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18990f30935f46ceb7c68b370f68b82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:36.918368 22942 tablet_service.cc:1467] Processing CreateTablet for tablet 18990f30935f46ceb7c68b370f68b82d (DEFAULT_TABLE table=test-table [id=31bbab7fb5c64f3b998c107a7187722f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:36.919312 22942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18990f30935f46ceb7c68b370f68b82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:36.926669 23315 tablet_bootstrap.cc:492] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c: Bootstrap starting.
I20260923 18:00:36.927176 23316 tablet_bootstrap.cc:492] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c: Bootstrap starting.
I20260923 18:00:36.929013 23316 tablet_bootstrap.cc:654] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:36.929090 23315 tablet_bootstrap.cc:654] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:36.929848 23315 log.cc:824] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:36.929891 23316 log.cc:824] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:36.931190 23317 tablet_bootstrap.cc:492] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b: Bootstrap starting.
I20260923 18:00:36.932183 23315 tablet_bootstrap.cc:492] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c: No bootstrap required, opened a new log
I20260923 18:00:36.932400 23315 ts_tablet_manager.cc:1397] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260923 18:00:36.933279 23317 tablet_bootstrap.cc:654] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:36.934047 23317 log.cc:824] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:36.936072 23317 tablet_bootstrap.cc:492] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b: No bootstrap required, opened a new log
I20260923 18:00:36.936285 23317 ts_tablet_manager.cc:1397] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260923 18:00:36.936213 23315 raft_consensus.cc:348] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } }
I20260923 18:00:36.936492 23315 raft_consensus.cc:374] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:36.936545 23315 raft_consensus.cc:729] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f248c4fde5a24ad28b187849e4eb225c, State: Initialized, Role: FOLLOWER
I20260923 18:00:36.936836 23316 tablet_bootstrap.cc:492] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c: No bootstrap required, opened a new log
I20260923 18:00:36.937027 23316 ts_tablet_manager.cc:1397] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260923 18:00:36.937292 23315 consensus_queue.cc:260] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } }
I20260923 18:00:36.938303 23315 ts_tablet_manager.cc:1428] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260923 18:00:36.938674 23282 heartbeater.cc:503] Master 127.15.139.190:41269 was elected leader, sending a full tablet report...
I20260923 18:00:36.942304 23316 raft_consensus.cc:348] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } }
I20260923 18:00:36.942605 23316 raft_consensus.cc:374] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:36.942663 23316 raft_consensus.cc:729] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b844678d1ab24411857fc79ae7010d2c, State: Initialized, Role: FOLLOWER
I20260923 18:00:36.943173 23316 consensus_queue.cc:260] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } }
I20260923 18:00:36.943926 23146 heartbeater.cc:503] Master 127.15.139.190:41269 was elected leader, sending a full tablet report...
I20260923 18:00:36.943746 23317 raft_consensus.cc:348] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } }
I20260923 18:00:36.944015 23317 raft_consensus.cc:374] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:36.944078 23317 raft_consensus.cc:729] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e53e7fbff400481f8979b8602fcd6a9b, State: Initialized, Role: FOLLOWER
I20260923 18:00:36.944320 23316 ts_tablet_manager.cc:1428] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260923 18:00:36.944571 23317 consensus_queue.cc:260] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } }
I20260923 18:00:36.947327 23317 ts_tablet_manager.cc:1428] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260923 18:00:36.947641 23009 heartbeater.cc:503] Master 127.15.139.190:41269 was elected leader, sending a full tablet report...
I20260923 18:00:37.083171 23321 raft_consensus.cc:482] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:37.083405 23321 raft_consensus.cc:504] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } }
I20260923 18:00:37.084717 23321 leader_election.cc:288] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b844678d1ab24411857fc79ae7010d2c (127.15.139.130:36899), e53e7fbff400481f8979b8602fcd6a9b (127.15.139.129:46199)
I20260923 18:00:37.090003 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:37.085043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:41947 (local address 127.15.139.130:36899)
0923 18:00:37.085189 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:37.085194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:37.085243 (+ 49us) server_negotiation.cc:407] Connection header received
0923 18:00:37.085362 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:37.085365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:37.085456 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:37.085555 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:37.085839 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.086737 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.088629 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.089287 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.089392 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:37.089441 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:37.089795 (+ 354us) server_negotiation.cc:299] Negotiation successful
0923 18:00:37.089871 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 18:00:37.090003 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:37.085344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:55207 (local address 127.15.139.129:46199)
0923 18:00:37.085538 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:37.085545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:37.085559 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:00:37.085608 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:37.085611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:37.085661 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:37.085731 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:37.085987 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.086749 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.088634 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.089258 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.089390 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:37.089441 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:37.089795 (+ 354us) server_negotiation.cc:299] Negotiation successful
0923 18:00:37.089875 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260923 18:00:37.090085 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:37.085105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:46199 (local address 127.15.139.131:55207)
0923 18:00:37.085195 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:37.085208 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:37.085269 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:37.085764 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:37.085768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:37.085788 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:37.085883 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:37.085887 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.086862 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.086865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:37.088497 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:37.088507 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.089394 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.089398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:37.089618 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:37.089685 (+ 67us) client_negotiation.cc:769] Sending connection context
0923 18:00:37.089767 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 18:00:37.089921 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":338}
I20260923 18:00:37.090102 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:37.084910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:36899 (local address 127.15.139.131:41947)
0923 18:00:37.085042 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:37.085071 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:37.085174 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:37.085576 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:37.085579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:37.085596 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:37.085695 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:37.085699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.086862 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.086865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:37.088502 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:37.088507 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.089394 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.089398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:37.089596 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:37.089686 (+ 90us) client_negotiation.cc:769] Sending connection context
0923 18:00:37.089767 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 18:00:37.089921 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":262}
I20260923 18:00:37.090857 22962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18990f30935f46ceb7c68b370f68b82d" candidate_uuid: "f248c4fde5a24ad28b187849e4eb225c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53e7fbff400481f8979b8602fcd6a9b" is_pre_election: true
I20260923 18:00:37.091085 23099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18990f30935f46ceb7c68b370f68b82d" candidate_uuid: "f248c4fde5a24ad28b187849e4eb225c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b844678d1ab24411857fc79ae7010d2c" is_pre_election: true
I20260923 18:00:37.091163 22962 raft_consensus.cc:2454] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f248c4fde5a24ad28b187849e4eb225c in term 0.
I20260923 18:00:37.091400 23099 raft_consensus.cc:2454] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f248c4fde5a24ad28b187849e4eb225c in term 0.
I20260923 18:00:37.091672 23169 leader_election.cc:302] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e53e7fbff400481f8979b8602fcd6a9b, f248c4fde5a24ad28b187849e4eb225c; no voters:
I20260923 18:00:37.091992 23321 raft_consensus.cc:2781] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:37.092104 23321 raft_consensus.cc:482] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:37.092162 23321 raft_consensus.cc:3037] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:37.093221 23321 raft_consensus.cc:504] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } }
I20260923 18:00:37.093636 23321 leader_election.cc:288] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [CANDIDATE]: Term 1 election: Requested vote from peers b844678d1ab24411857fc79ae7010d2c (127.15.139.130:36899), e53e7fbff400481f8979b8602fcd6a9b (127.15.139.129:46199)
I20260923 18:00:37.094069 23099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18990f30935f46ceb7c68b370f68b82d" candidate_uuid: "f248c4fde5a24ad28b187849e4eb225c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b844678d1ab24411857fc79ae7010d2c"
I20260923 18:00:37.094132 22962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18990f30935f46ceb7c68b370f68b82d" candidate_uuid: "f248c4fde5a24ad28b187849e4eb225c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53e7fbff400481f8979b8602fcd6a9b"
I20260923 18:00:37.094225 23099 raft_consensus.cc:3037] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:37.094290 22962 raft_consensus.cc:3037] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:37.095489 22962 raft_consensus.cc:2454] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f248c4fde5a24ad28b187849e4eb225c in term 1.
I20260923 18:00:37.095480 23099 raft_consensus.cc:2454] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f248c4fde5a24ad28b187849e4eb225c in term 1.
I20260923 18:00:37.095837 23169 leader_election.cc:302] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e53e7fbff400481f8979b8602fcd6a9b, f248c4fde5a24ad28b187849e4eb225c; no voters:
I20260923 18:00:37.096086 23321 raft_consensus.cc:2781] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:37.096308 23321 raft_consensus.cc:686] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 1 LEADER]: Becoming Leader. State: Replica: f248c4fde5a24ad28b187849e4eb225c, State: Running, Role: LEADER
I20260923 18:00:37.096678 23321 consensus_queue.cc:237] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } }
I20260923 18:00:37.100433 22037 catalog_manager.cc:5495] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c reported cstate change: term changed from 0 to 1, leader changed from <none> to f248c4fde5a24ad28b187849e4eb225c (127.15.139.131). New cstate: current_term: 1 leader_uuid: "f248c4fde5a24ad28b187849e4eb225c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } health_report { overall_health: HEALTHY } } }
W20260923 18:00:37.105264 23284 tablet.cc:2369] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:00:37.135293 23148 tablet.cc:2369] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:00:37.156329 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:37.151861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.15.139.131:45875)
0923 18:00:37.152032 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:37.152039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:37.152063 (+ 24us) server_negotiation.cc:407] Connection header received
0923 18:00:37.152269 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:37.152272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:37.152337 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:37.152507 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:37.152816 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.153531 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.154517 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.154767 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.154909 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:37.155286 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:37.155827 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:37.155992 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:37.156061 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 18:00:37.156131 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 18:00:37.162868 22962 raft_consensus.cc:1261] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b [term 1 FOLLOWER]: Refusing update from remote peer f248c4fde5a24ad28b187849e4eb225c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:37.162963 23099 raft_consensus.cc:1261] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c [term 1 FOLLOWER]: Refusing update from remote peer f248c4fde5a24ad28b187849e4eb225c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:37.163650 23324 consensus_queue.cc:1035] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:37.163900 23321 consensus_queue.cc:1035] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 18:00:37.173199 23011 tablet.cc:2369] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:00:37.174989 23329 mvcc.cc:205] Tried to move back new op lower bound from 7332603646611042304 to 7332603646350864384. Current Snapshot: MvccSnapshot[applied={T|T < 7332603646611042304}]
I20260923 18:00:37.178561 23328 mvcc.cc:205] Tried to move back new op lower bound from 7332603646611042304 to 7332603646350864384. Current Snapshot: MvccSnapshot[applied={T|T < 7332603646611042304}]
I20260923 18:00:37.182349 22037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:37.182505 22037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:37.188549 22037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be70c9d1f51f405c8c9e59aa78004870: Sending DeleteTablet for 3 replicas of tablet 18990f30935f46ceb7c68b370f68b82d
I20260923 18:00:37.189215 23079 tablet_service.cc:1514] Processing DeleteTablet for tablet 18990f30935f46ceb7c68b370f68b82d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:37 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:59720
I20260923 18:00:37.189224 22942 tablet_service.cc:1514] Processing DeleteTablet for tablet 18990f30935f46ceb7c68b370f68b82d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:37 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:53242
I20260923 18:00:37.189700 23337 tablet_replica.cc:330] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b: stopping tablet replica
I20260923 18:00:37.189682 23336 tablet_replica.cc:330] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c: stopping tablet replica
I20260923 18:00:37.190086 23336 raft_consensus.cc:2229] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:37.190086 23337 raft_consensus.cc:2229] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:37.190454 23336 raft_consensus.cc:2258] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:37.190496 23337 raft_consensus.cc:2258] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:37.192219 23336 ts_tablet_manager.cc:1905] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:37.192451 23337 ts_tablet_manager.cc:1905] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:37.192975 23215 tablet_service.cc:1514] Processing DeleteTablet for tablet 18990f30935f46ceb7c68b370f68b82d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:37 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:56234
I20260923 18:00:37.193692 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 22005
I20260923 18:00:37.196096 23336 ts_tablet_manager.cc:1918] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:37.196210 23336 log.cc:1196] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/18990f30935f46ceb7c68b370f68b82d
I20260923 18:00:37.196242 23337 ts_tablet_manager.cc:1918] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:37.196483 23337 log.cc:1196] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/18990f30935f46ceb7c68b370f68b82d
I20260923 18:00:37.196609 23336 ts_tablet_manager.cc:1939] T 18990f30935f46ceb7c68b370f68b82d P b844678d1ab24411857fc79ae7010d2c: Deleting consensus metadata
I20260923 18:00:37.196950 23337 ts_tablet_manager.cc:1939] T 18990f30935f46ceb7c68b370f68b82d P e53e7fbff400481f8979b8602fcd6a9b: Deleting consensus metadata
I20260923 18:00:37.200289 23339 tablet_replica.cc:330] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c: stopping tablet replica
I20260923 18:00:37.200732 23339 raft_consensus.cc:2229] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:37.201195 23339 raft_consensus.cc:2258] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 18:00:37.202244 23034 connection.cc:535] server connection from 127.0.0.1:59720 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 18:00:37.202685 22898 connection.cc:535] server connection from 127.0.0.1:53242 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 18:00:37.203028 23339 ts_tablet_manager.cc:1905] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:37.206135 23339 ts_tablet_manager.cc:1918] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:37.206300 23339 log.cc:1196] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/18990f30935f46ceb7c68b370f68b82d
I20260923 18:00:37.206761 23339 ts_tablet_manager.cc:1939] T 18990f30935f46ceb7c68b370f68b82d P f248c4fde5a24ad28b187849e4eb225c: Deleting consensus metadata
W20260923 18:00:37.208196 23171 connection.cc:441] server connection from 127.0.0.1:56234 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56234 (request call id 1) could send its response
I20260923 18:00:37.211189 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:37.191527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42018 (local address 127.15.139.189:44001)
0923 18:00:37.191648 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:37.191654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:37.199905 (+ 8251us) server_negotiation.cc:407] Connection header received
0923 18:00:37.200070 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:37.200077 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:37.200157 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:37.200237 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:37.201351 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.202143 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.204080 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.204445 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.204530 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:37.207245 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:37.207249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:37.207252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:37.207278 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:37.208976 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:37.209751 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:37.209755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:37.209757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:37.209843 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:37.210169 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:37.210172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:37.210174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:37.210386 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:37.210487 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:37.210816 (+ 329us) server_negotiation.cc:299] Negotiation successful
0923 18:00:37.210941 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 18:00:37.217173 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:37.206366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.15.139.188:37763)
0923 18:00:37.206507 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:37.206511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:37.207916 (+ 1405us) server_negotiation.cc:407] Connection header received
0923 18:00:37.208114 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:37.208118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:37.208172 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:37.208228 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:37.209141 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.209958 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.211082 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.211393 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.211470 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:37.213528 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:37.213531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:37.213534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:37.213577 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:37.215328 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:37.215856 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:37.215859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:37.215861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:37.215946 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:37.216221 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:37.216224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:37.216225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:37.216427 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:37.216537 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:37.216884 (+ 347us) server_negotiation.cc:299] Negotiation successful
0923 18:00:37.217012 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 18:00:37.219435 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:37.207093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42028 (local address 127.15.139.189:44001)
0923 18:00:37.207227 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:37.207231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:37.209332 (+ 2101us) server_negotiation.cc:407] Connection header received
0923 18:00:37.209521 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:37.209524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:37.209572 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:37.209627 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:37.210489 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.211320 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.212675 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.213040 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.213114 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:37.214915 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:37.214919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:37.214922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:37.214948 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:37.216813 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:37.217442 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:37.217446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:37.217448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:37.217538 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:37.217981 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:37.217985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:37.217987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:37.218203 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:37.218309 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:37.219111 (+ 802us) server_negotiation.cc:299] Negotiation successful
0923 18:00:37.219232 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 18:00:37.221788 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:37.191636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55952 (local address 127.15.139.188:37763)
0923 18:00:37.191747 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:37.191752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:37.213032 (+ 21280us) server_negotiation.cc:407] Connection header received
0923 18:00:37.213232 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:37.213235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:37.213280 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:37.213332 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:37.214495 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.215334 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.216456 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:37.216782 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:37.216851 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:37.218379 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:37.218383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:37.218386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:37.218409 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:37.220100 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:37.220566 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:37.220570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:37.220572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:37.220665 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:37.220932 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:37.220935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:37.220937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:37.221148 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:37.221275 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:37.221475 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 18:00:37.221597 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43,"spinlock_wait_cycles":2944}
I20260923 18:00:38.177630 23347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.176771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.131:54379)
0923 18:00:38.177346 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.177489 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":173,"threads_started":1}
W20260923 18:00:38.178079 23282 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:41269 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
I20260923 18:00:38.180546 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.180151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.129:51587)
0923 18:00:38.180376 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.180464 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":69,"threads_started":1}
W20260923 18:00:38.180850 23009 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:41269 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
I20260923 18:00:38.182271 23349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.181877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.130:60643)
0923 18:00:38.182132 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.182200 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":116,"threads_started":1}
W20260923 18:00:38.182559 23146 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:41269 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
I20260923 18:00:38.688079 23350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be70c9d1f51f405c8c9e59aa78004870)
I20260923 18:00:38.688138 23351 raft_consensus.cc:482] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be70c9d1f51f405c8c9e59aa78004870)
I20260923 18:00:38.688236 23350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } }
I20260923 18:00:38.688438 23351 raft_consensus.cc:504] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } }
I20260923 18:00:38.689037 23350 leader_election.cc:288] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269), 460156afb37e44d5a0c97e63447c0c7c (127.15.139.189:44001)
I20260923 18:00:38.689723 23351 leader_election.cc:288] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269), 647b766238a14c569a467052c41b4ec1 (127.15.139.188:37763)
I20260923 18:00:38.690052 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.689099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42058)
0923 18:00:38.689721 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.689832 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":71,"threads_started":1}
I20260923 18:00:38.690289 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.689761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42062)
0923 18:00:38.690010 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.690182 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":63,"threads_started":1}
W20260923 18:00:38.690323 22816 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
W20260923 18:00:38.690521 22626 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
I20260923 18:00:38.691367 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.691088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42070)
0923 18:00:38.691225 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.691300 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260923 18:00:38.691475 22816 leader_election.cc:334] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269): Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
I20260923 18:00:38.691622 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.691255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42078)
0923 18:00:38.691377 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.691420 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260923 18:00:38.691767 22626 leader_election.cc:334] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269): Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
I20260923 18:00:38.694162 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.689331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44001 (local address 127.0.0.1:42034)
0923 18:00:38.690184 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.690198 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:38.690309 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:38.690727 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:38.690733 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:38.690757 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:38.690868 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:38.690876 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:38.691957 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:38.691960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:38.692813 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:38.692822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:38.693613 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:38.693616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:38.693775 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:38.693802 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 18:00:38.693883 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 18:00:38.693957 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"mutex_wait_us":50,"thread_start_us":65,"threads_started":1}
I20260923 18:00:38.694249 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.689414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42034 (local address 127.15.139.189:44001)
0923 18:00:38.689724 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:38.689730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:38.690403 (+ 673us) server_negotiation.cc:407] Connection header received
0923 18:00:38.690456 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:38.690459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:38.690503 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:38.690596 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:38.691003 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:38.691829 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:38.692944 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:38.693494 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:38.693592 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:38.693640 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:38.694020 (+ 380us) server_negotiation.cc:299] Negotiation successful
0923 18:00:38.694127 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":89,"threads_started":1}
I20260923 18:00:38.694644 23356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.690080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55972 (local address 127.15.139.188:37763)
0923 18:00:38.690785 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:38.690791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:38.690807 (+ 16us) server_negotiation.cc:407] Connection header received
0923 18:00:38.690855 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:38.690859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:38.690922 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:38.691322 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:38.691428 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:38.692159 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:38.693353 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:38.694005 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:38.694160 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:38.694221 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:38.694438 (+ 217us) server_negotiation.cc:299] Negotiation successful
0923 18:00:38.694511 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":633,"thread_start_us":68,"threads_started":1}
I20260923 18:00:38.694657 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.690278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:37763 (local address 127.0.0.1:55972)
0923 18:00:38.690552 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.690572 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:38.690673 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:38.691064 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:38.691068 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:38.691087 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:38.691202 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:38.691209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:38.692282 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:38.692289 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:38.693234 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:38.693244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:38.694191 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:38.694195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:38.694295 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:38.694327 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 18:00:38.694422 (+ 95us) client_negotiation.cc:240] Negotiation successful
0923 18:00:38.694524 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":60,"threads_started":1}
I20260923 18:00:38.694823 22648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "647b766238a14c569a467052c41b4ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "460156afb37e44d5a0c97e63447c0c7c" is_pre_election: true
I20260923 18:00:38.694991 22648 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647b766238a14c569a467052c41b4ec1 in term 1.
I20260923 18:00:38.695063 22839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "460156afb37e44d5a0c97e63447c0c7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "647b766238a14c569a467052c41b4ec1" is_pre_election: true
I20260923 18:00:38.695272 22839 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 460156afb37e44d5a0c97e63447c0c7c in term 1.
I20260923 18:00:38.695350 22814 leader_election.cc:302] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 460156afb37e44d5a0c97e63447c0c7c, 647b766238a14c569a467052c41b4ec1; no voters: be70c9d1f51f405c8c9e59aa78004870
I20260923 18:00:38.695609 23350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 18:00:38.695670 23350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader be70c9d1f51f405c8c9e59aa78004870)
I20260923 18:00:38.695721 23350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260923 18:00:38.695688 22625 leader_election.cc:302] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 460156afb37e44d5a0c97e63447c0c7c, 647b766238a14c569a467052c41b4ec1; no voters: be70c9d1f51f405c8c9e59aa78004870
I20260923 18:00:38.695922 23351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 18:00:38.695984 23351 raft_consensus.cc:482] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 FOLLOWER]: Starting leader election (detected failure of leader be70c9d1f51f405c8c9e59aa78004870)
I20260923 18:00:38.696033 23351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 1 FOLLOWER]: Advancing to term 2
I20260923 18:00:38.697057 23350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } }
I20260923 18:00:38.697194 23351 raft_consensus.cc:504] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } }
I20260923 18:00:38.697448 23350 leader_election.cc:288] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 2 election: Requested vote from peers be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269), 460156afb37e44d5a0c97e63447c0c7c (127.15.139.189:44001)
I20260923 18:00:38.697517 23351 leader_election.cc:288] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [CANDIDATE]: Term 2 election: Requested vote from peers be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269), 647b766238a14c569a467052c41b4ec1 (127.15.139.188:37763)
I20260923 18:00:38.697814 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.697576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42088)
0923 18:00:38.697697 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.697750 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260923 18:00:38.697976 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.697734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42092)
0923 18:00:38.697840 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.697917 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260923 18:00:38.697980 22648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "647b766238a14c569a467052c41b4ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "460156afb37e44d5a0c97e63447c0c7c"
I20260923 18:00:38.698184 22648 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 647b766238a14c569a467052c41b4ec1 in current term 2: Already voted for candidate 460156afb37e44d5a0c97e63447c0c7c in this term.
I20260923 18:00:38.698278 22839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "460156afb37e44d5a0c97e63447c0c7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "647b766238a14c569a467052c41b4ec1"
I20260923 18:00:38.698442 22839 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 460156afb37e44d5a0c97e63447c0c7c in current term 2: Already voted for candidate 647b766238a14c569a467052c41b4ec1 in this term.
I20260923 18:00:38.698848 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.698423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42106)
0923 18:00:38.698739 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.698786 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208}
I20260923 18:00:38.698930 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:38.698719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42108)
0923 18:00:38.698807 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:38.698855 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260923 18:00:38.699026 22626 leader_election.cc:334] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269): Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
W20260923 18:00:38.699059 22816 leader_election.cc:334] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269): Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
I20260923 18:00:38.699105 22626 leader_election.cc:302] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 460156afb37e44d5a0c97e63447c0c7c; no voters: 647b766238a14c569a467052c41b4ec1, be70c9d1f51f405c8c9e59aa78004870
I20260923 18:00:38.699154 22816 leader_election.cc:302] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647b766238a14c569a467052c41b4ec1; no voters: 460156afb37e44d5a0c97e63447c0c7c, be70c9d1f51f405c8c9e59aa78004870
I20260923 18:00:38.699239 23351 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260923 18:00:38.699298 23350 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260923 18:00:39.179263 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:39.178734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.131:39263)
0923 18:00:39.179080 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:39.179150 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":98,"threads_started":1}
I20260923 18:00:39.181814 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:39.181453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.129:51707)
0923 18:00:39.181684 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:39.181737 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":72,"threads_started":1}
I20260923 18:00:39.183456 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:39.183091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.130:51183)
0923 18:00:39.183326 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:39.183379 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260923 18:00:40.180676 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.180179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.131:49147)
0923 18:00:40.180479 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.180545 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":78,"threads_started":1}
I20260923 18:00:40.182796 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.182428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.129:53123)
0923 18:00:40.182666 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.182720 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":61,"threads_started":1}
I20260923 18:00:40.184672 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.184285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.130:55549)
0923 18:00:40.184522 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.184598 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":55,"threads_started":1}
I20260923 18:00:40.604323 23366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:40.604527 23366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } }
I20260923 18:00:40.605003 23366 leader_election.cc:288] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269), 460156afb37e44d5a0c97e63447c0c7c (127.15.139.189:44001)
I20260923 18:00:40.605433 22648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "647b766238a14c569a467052c41b4ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "460156afb37e44d5a0c97e63447c0c7c" is_pre_election: true
I20260923 18:00:40.605624 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.605217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42224)
0923 18:00:40.605444 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.605504 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260923 18:00:40.605638 22648 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647b766238a14c569a467052c41b4ec1 in term 2.
I20260923 18:00:40.605973 22814 leader_election.cc:302] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 460156afb37e44d5a0c97e63447c0c7c, 647b766238a14c569a467052c41b4ec1; no voters:
I20260923 18:00:40.606196 23366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260923 18:00:40.606281 23366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:40.606323 23366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260923 18:00:40.606468 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.606226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42238)
0923 18:00:40.606343 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.606403 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 18:00:40.606583 22816 leader_election.cc:334] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269): Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
I20260923 18:00:40.607463 23366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } }
I20260923 18:00:40.607937 23366 leader_election.cc:288] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 3 election: Requested vote from peers be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269), 460156afb37e44d5a0c97e63447c0c7c (127.15.139.189:44001)
I20260923 18:00:40.608260 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.608081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42240)
0923 18:00:40.608165 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.608210 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260923 18:00:40.608361 22648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "647b766238a14c569a467052c41b4ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "460156afb37e44d5a0c97e63447c0c7c"
I20260923 18:00:40.608484 22648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 2 FOLLOWER]: Advancing to term 3
I20260923 18:00:40.608903 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.608710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42256)
0923 18:00:40.608809 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.608858 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 18:00:40.609031 22816 leader_election.cc:334] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269): Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
I20260923 18:00:40.609543 22648 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647b766238a14c569a467052c41b4ec1 in term 3.
I20260923 18:00:40.609802 22814 leader_election.cc:302] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 460156afb37e44d5a0c97e63447c0c7c, 647b766238a14c569a467052c41b4ec1; no voters: be70c9d1f51f405c8c9e59aa78004870
I20260923 18:00:40.609942 23366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 18:00:40.610136 23366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [term 3 LEADER]: Becoming Leader. State: Replica: 647b766238a14c569a467052c41b4ec1, State: Running, Role: LEADER
I20260923 18:00:40.610415 23366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } }
I20260923 18:00:40.611428 23369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 647b766238a14c569a467052c41b4ec1. Latest consensus state: current_term: 3 leader_uuid: "647b766238a14c569a467052c41b4ec1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } } }
I20260923 18:00:40.611564 23369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:40.611879 23371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:40.614154 23371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:40.614560 23371 catalog_manager.cc:1259] Loaded cluster ID: c84c2c9cb9bd495391ee03b26f00e237
I20260923 18:00:40.614605 23371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:40.615156 23371 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:40.615602 23371 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: Loaded TSK: 0
I20260923 18:00:40.616286 23371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:00:40.635555 22829 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:00:40.638844 22829 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:40.641783 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.641543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42278)
0923 18:00:40.641675 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.641724 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260923 18:00:40.641860 22648 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [term 3 FOLLOWER]: Refusing update from remote peer 647b766238a14c569a467052c41b4ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260923 18:00:40.642393 23369 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260923 18:00:40.642571 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.642348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.0.0.1:42280)
0923 18:00:40.642455 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.642506 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260923 18:00:40.642671 22816 consensus_peers.cc:487] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 -> Peer be70c9d1f51f405c8c9e59aa78004870 (127.15.139.190:41269): Couldn't send request to peer be70c9d1f51f405c8c9e59aa78004870. Status: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 18:00:40.643970 23373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 647b766238a14c569a467052c41b4ec1. Latest consensus state: current_term: 3 leader_uuid: "647b766238a14c569a467052c41b4ec1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } } }
I20260923 18:00:40.644114 23373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:00:40.645215 23369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "647b766238a14c569a467052c41b4ec1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } } }
I20260923 18:00:40.645334 23369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:40.645574 23373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "647b766238a14c569a467052c41b4ec1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "be70c9d1f51f405c8c9e59aa78004870" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41269 } } peers { permanent_uuid: "460156afb37e44d5a0c97e63447c0c7c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44001 } attrs { promote: false } } peers { permanent_uuid: "647b766238a14c569a467052c41b4ec1" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 37763 } attrs { promote: false } } }
I20260923 18:00:40.645685 23373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460156afb37e44d5a0c97e63447c0c7c [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:00:40.646117 23374 mvcc.cc:205] Tried to move back new op lower bound from 7332603660864647168 to 7332603660742651904. Current Snapshot: MvccSnapshot[applied={T|T < 7332603660864647168 or (T in {7332603660864647168})}]
I20260923 18:00:40.655598 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.650774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:45875 (local address 127.0.0.1:56246)
0923 18:00:40.651178 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.651194 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:40.651288 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:40.651836 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:40.651838 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:40.651854 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:40.651990 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:40.651996 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.653027 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.653032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:40.654152 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:40.654161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.655070 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.655073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:40.655226 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:40.655264 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 18:00:40.655351 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 18:00:40.655429 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":19}
I20260923 18:00:40.655642 23379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.650893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56246 (local address 127.15.139.131:45875)
0923 18:00:40.651544 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:40.651550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:40.651567 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:00:40.651623 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:40.651626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:40.651680 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:40.652102 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:40.652225 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.652934 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.654269 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.654929 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.655212 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:40.655261 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:40.655366 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 18:00:40.655442 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":100,"threads_started":1}
I20260923 18:00:40.655879 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.651216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53254 (local address 127.15.139.129:46199)
0923 18:00:40.651756 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:40.651761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:40.651775 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:00:40.651862 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:40.651864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:40.651903 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:40.651976 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:40.652215 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.652940 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.654455 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.655088 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.655207 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:40.655261 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:40.655653 (+ 392us) server_negotiation.cc:299] Negotiation successful
0923 18:00:40.655747 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":80,"threads_started":1}
I20260923 18:00:40.656456 23215 tablet_service.cc:1467] Processing CreateTablet for tablet 3b6381949cb4475490c01a7cfa8c5e20 (DEFAULT_TABLE table=test-table [id=987a3b0277b848f99989cf04fabe3258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:40.656698 23382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.651049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:46199 (local address 127.0.0.1:53254)
0923 18:00:40.651675 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.651689 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:40.651773 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:40.652005 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:40.652007 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:40.652024 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:40.652115 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:40.652120 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.653110 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.653113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:40.654321 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:40.654331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.655316 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.655322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:40.655409 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:40.655431 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 18:00:40.655522 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 18:00:40.655590 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260923 18:00:40.656796 23215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b6381949cb4475490c01a7cfa8c5e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:40.657352 22942 tablet_service.cc:1467] Processing CreateTablet for tablet 3b6381949cb4475490c01a7cfa8c5e20 (DEFAULT_TABLE table=test-table [id=987a3b0277b848f99989cf04fabe3258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:40.657634 22942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b6381949cb4475490c01a7cfa8c5e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:40.657701 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.651370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:36899 (local address 127.0.0.1:59726)
0923 18:00:40.653139 (+ 1769us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.653152 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:40.653240 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:40.653500 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:40.653503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:40.653522 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:40.653600 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:40.653605 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.654523 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.654528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:40.656444 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:40.656453 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.657320 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.657324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:40.657408 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:40.657430 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 18:00:40.657517 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 18:00:40.657580 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1711,"mutex_wait_us":23,"thread_start_us":96,"threads_started":1}
I20260923 18:00:40.657776 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.651473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59726 (local address 127.15.139.130:36899)
0923 18:00:40.651732 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:40.651736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:40.653236 (+ 1500us) server_negotiation.cc:407] Connection header received
0923 18:00:40.653325 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:40.653328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:40.653379 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:40.653466 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:40.653715 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.654445 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.656574 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.657188 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.657311 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:40.657359 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:40.657550 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 18:00:40.657646 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":78,"threads_started":1}
I20260923 18:00:40.659457 23384 tablet_bootstrap.cc:492] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c: Bootstrap starting.
I20260923 18:00:40.660019 23385 tablet_bootstrap.cc:492] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b: Bootstrap starting.
I20260923 18:00:40.660244 23079 tablet_service.cc:1467] Processing CreateTablet for tablet 3b6381949cb4475490c01a7cfa8c5e20 (DEFAULT_TABLE table=test-table [id=987a3b0277b848f99989cf04fabe3258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:40.660531 23079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b6381949cb4475490c01a7cfa8c5e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:40.660543 23384 tablet_bootstrap.cc:654] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:40.661027 23385 tablet_bootstrap.cc:654] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:40.661701 23384 tablet_bootstrap.cc:492] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c: No bootstrap required, opened a new log
I20260923 18:00:40.661873 23384 ts_tablet_manager.cc:1397] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:40.662261 23385 tablet_bootstrap.cc:492] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b: No bootstrap required, opened a new log
I20260923 18:00:40.662248 23384 raft_consensus.cc:348] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } }
I20260923 18:00:40.662397 23384 raft_consensus.cc:374] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:40.662467 23384 raft_consensus.cc:729] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f248c4fde5a24ad28b187849e4eb225c, State: Initialized, Role: FOLLOWER
I20260923 18:00:40.662660 23384 consensus_queue.cc:260] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } }
I20260923 18:00:40.663048 23384 ts_tablet_manager.cc:1428] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:40.663126 23385 ts_tablet_manager.cc:1397] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:40.663354 23389 tablet_bootstrap.cc:492] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c: Bootstrap starting.
I20260923 18:00:40.663738 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.663452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.131:56065)
0923 18:00:40.663618 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.663673 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260923 18:00:40.663705 23385 raft_consensus.cc:348] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } }
I20260923 18:00:40.663813 23385 raft_consensus.cc:374] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:40.663851 23385 raft_consensus.cc:729] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e53e7fbff400481f8979b8602fcd6a9b, State: Initialized, Role: FOLLOWER
I20260923 18:00:40.664357 23389 tablet_bootstrap.cc:654] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:40.664006 23385 consensus_queue.cc:260] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } }
I20260923 18:00:40.664770 23385 ts_tablet_manager.cc:1428] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:40.665422 23281 heartbeater.cc:503] Master 127.15.139.188:37763 was elected leader, sending a full tablet report...
I20260923 18:00:40.665526 23389 tablet_bootstrap.cc:492] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c: No bootstrap required, opened a new log
I20260923 18:00:40.665601 23389 ts_tablet_manager.cc:1397] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:40.666133 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.665814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.129:43575)
0923 18:00:40.666006 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.666062 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260923 18:00:40.666064 23389 raft_consensus.cc:348] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } }
I20260923 18:00:40.666177 23389 raft_consensus.cc:374] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:40.666210 23389 raft_consensus.cc:729] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b844678d1ab24411857fc79ae7010d2c, State: Initialized, Role: FOLLOWER
I20260923 18:00:40.666347 23389 consensus_queue.cc:260] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } }
I20260923 18:00:40.666728 23389 ts_tablet_manager.cc:1428] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:40.666774 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.666514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.131:37717)
0923 18:00:40.666666 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.666714 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260923 18:00:40.667438 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.667142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.130:56463)
0923 18:00:40.667319 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.667369 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260923 18:00:40.668469 23008 heartbeater.cc:503] Master 127.15.139.188:37763 was elected leader, sending a full tablet report...
I20260923 18:00:40.668560 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.668320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.130:46419)
0923 18:00:40.668439 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.668498 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260923 18:00:40.669509 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.669227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.129:53155)
0923 18:00:40.669364 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.669417 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260923 18:00:40.670706 23145 heartbeater.cc:503] Master 127.15.139.188:37763 was elected leader, sending a full tablet report...
I20260923 18:00:40.694289 23388 raft_consensus.cc:482] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:40.694439 23388 raft_consensus.cc:504] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } }
I20260923 18:00:40.694936 23388 leader_election.cc:288] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e53e7fbff400481f8979b8602fcd6a9b (127.15.139.129:46199), b844678d1ab24411857fc79ae7010d2c (127.15.139.130:36899)
I20260923 18:00:40.695533 22962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b6381949cb4475490c01a7cfa8c5e20" candidate_uuid: "f248c4fde5a24ad28b187849e4eb225c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53e7fbff400481f8979b8602fcd6a9b" is_pre_election: true
I20260923 18:00:40.695636 23099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b6381949cb4475490c01a7cfa8c5e20" candidate_uuid: "f248c4fde5a24ad28b187849e4eb225c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b844678d1ab24411857fc79ae7010d2c" is_pre_election: true
I20260923 18:00:40.695730 22962 raft_consensus.cc:2454] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f248c4fde5a24ad28b187849e4eb225c in term 0.
I20260923 18:00:40.695797 23099 raft_consensus.cc:2454] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f248c4fde5a24ad28b187849e4eb225c in term 0.
I20260923 18:00:40.696153 23170 leader_election.cc:302] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b844678d1ab24411857fc79ae7010d2c, f248c4fde5a24ad28b187849e4eb225c; no voters:
I20260923 18:00:40.696341 23388 raft_consensus.cc:2781] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:40.696401 23388 raft_consensus.cc:482] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:40.696429 23388 raft_consensus.cc:3037] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:40.697458 23388 raft_consensus.cc:504] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } }
I20260923 18:00:40.697826 23388 leader_election.cc:288] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [CANDIDATE]: Term 1 election: Requested vote from peers e53e7fbff400481f8979b8602fcd6a9b (127.15.139.129:46199), b844678d1ab24411857fc79ae7010d2c (127.15.139.130:36899)
I20260923 18:00:40.698203 23099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b6381949cb4475490c01a7cfa8c5e20" candidate_uuid: "f248c4fde5a24ad28b187849e4eb225c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b844678d1ab24411857fc79ae7010d2c"
I20260923 18:00:40.698237 22962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b6381949cb4475490c01a7cfa8c5e20" candidate_uuid: "f248c4fde5a24ad28b187849e4eb225c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53e7fbff400481f8979b8602fcd6a9b"
I20260923 18:00:40.698318 23099 raft_consensus.cc:3037] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:40.698391 22962 raft_consensus.cc:3037] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:40.699652 22962 raft_consensus.cc:2454] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f248c4fde5a24ad28b187849e4eb225c in term 1.
I20260923 18:00:40.699673 23099 raft_consensus.cc:2454] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f248c4fde5a24ad28b187849e4eb225c in term 1.
I20260923 18:00:40.699975 23170 leader_election.cc:302] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b844678d1ab24411857fc79ae7010d2c, f248c4fde5a24ad28b187849e4eb225c; no voters:
I20260923 18:00:40.700194 23388 raft_consensus.cc:2781] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:40.700436 23388 raft_consensus.cc:686] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 1 LEADER]: Becoming Leader. State: Replica: f248c4fde5a24ad28b187849e4eb225c, State: Running, Role: LEADER
I20260923 18:00:40.700649 23388 consensus_queue.cc:237] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } } peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } }
I20260923 18:00:40.701207 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.700937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.131:52923)
0923 18:00:40.701062 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.701125 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 18:00:40.702422 22829 catalog_manager.cc:5495] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c reported cstate change: term changed from 0 to 1, leader changed from <none> to f248c4fde5a24ad28b187849e4eb225c (127.15.139.131). New cstate: current_term: 1 leader_uuid: "f248c4fde5a24ad28b187849e4eb225c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f248c4fde5a24ad28b187849e4eb225c" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 } health_report { overall_health: UNKNOWN } } }
I20260923 18:00:40.720227 23379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.717358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56256 (local address 127.15.139.131:45875)
0923 18:00:40.717509 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:40.717512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:40.717525 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:00:40.717569 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:40.717571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:40.717620 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:40.717697 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:40.717915 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.718412 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.719224 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:40.719433 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:40.719596 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:40.719785 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:40.719847 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:40.719900 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:40.719997 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 18:00:40.720041 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260923 18:00:40.722347 23099 raft_consensus.cc:1261] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c [term 1 FOLLOWER]: Refusing update from remote peer f248c4fde5a24ad28b187849e4eb225c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:40.722347 22962 raft_consensus.cc:1261] T 3b6381949cb4475490c01a7cfa8c5e20 P e53e7fbff400481f8979b8602fcd6a9b [term 1 FOLLOWER]: Refusing update from remote peer f248c4fde5a24ad28b187849e4eb225c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:40.722800 23388 consensus_queue.cc:1035] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b844678d1ab24411857fc79ae7010d2c" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 36899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:40.723044 23393 consensus_queue.cc:1035] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e53e7fbff400481f8979b8602fcd6a9b" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 46199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:40.725013 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.724660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.130:56277)
0923 18:00:40.724873 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.724930 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260923 18:00:40.725322 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.724915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.129:55581)
0923 18:00:40.725199 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.725255 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220}
I20260923 18:00:40.727651 23396 mvcc.cc:205] Tried to move back new op lower bound from 7332603661194186752 to 7332603661111164928. Current Snapshot: MvccSnapshot[applied={T|T < 7332603661194186752 or (T in {7332603661194186752})}]
I20260923 18:00:40.727913 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.727656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.131:49007)
0923 18:00:40.727788 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.727840 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260923 18:00:40.728940 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.728691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.131:58203)
0923 18:00:40.728831 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.728882 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260923 18:00:40.730664 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.730364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.130:34869)
0923 18:00:40.730559 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.730599 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260923 18:00:40.731010 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.730704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:41269 (local address 127.15.139.129:51729)
0923 18:00:40.730885 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.730938 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260923 18:00:40.732466 22828 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:40.732653 22828 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:40.736905 22828 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 647b766238a14c569a467052c41b4ec1: Sending DeleteTablet for 3 replicas of tablet 3b6381949cb4475490c01a7cfa8c5e20
I20260923 18:00:40.737550 22942 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b6381949cb4475490c01a7cfa8c5e20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:40 UTC) from {username='oryx', principal='oryx/127.15.139.188@KRBTEST.COM'} at 127.0.0.1:53254
I20260923 18:00:40.737596 23215 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b6381949cb4475490c01a7cfa8c5e20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:40 UTC) from {username='oryx', principal='oryx/127.15.139.188@KRBTEST.COM'} at 127.0.0.1:56246
I20260923 18:00:40.737900 23214 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b6381949cb4475490c01a7cfa8c5e20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:40 UTC) from {username='oryx', principal='oryx/127.15.139.188@KRBTEST.COM'} at 127.0.0.1:56246
I20260923 18:00:40.737926 23405 tablet_replica.cc:330] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c: stopping tablet replica
I20260923 18:00:40.738047 23405 raft_consensus.cc:2229] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:40.738093 23079 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b6381949cb4475490c01a7cfa8c5e20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:40 UTC) from {username='oryx', principal='oryx/127.15.139.188@KRBTEST.COM'} at 127.0.0.1:59726
I20260923 18:00:40.738191 23405 raft_consensus.cc:2258] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:40.738466 23406 tablet_replica.cc:330] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c: stopping tablet replica
I20260923 18:00:40.738601 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 22783
I20260923 18:00:40.738611 23406 raft_consensus.cc:2229] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:40.739374 23406 raft_consensus.cc:2258] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:40.740248 23405 ts_tablet_manager.cc:1905] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:40.742177 23405 ts_tablet_manager.cc:1918] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:40.742244 23405 log.cc:1196] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/3b6381949cb4475490c01a7cfa8c5e20
I20260923 18:00:40.742527 23405 ts_tablet_manager.cc:1939] T 3b6381949cb4475490c01a7cfa8c5e20 P f248c4fde5a24ad28b187849e4eb225c: Deleting consensus metadata
I20260923 18:00:40.743259 22815 catalog_manager.cc:4841] TS f248c4fde5a24ad28b187849e4eb225c (127.15.139.131:45875): tablet 3b6381949cb4475490c01a7cfa8c5e20 (table test-table [id=987a3b0277b848f99989cf04fabe3258]) successfully deleted
W20260923 18:00:40.743625 22815 catalog_manager.cc:4808] TS f248c4fde5a24ad28b187849e4eb225c (127.15.139.131:45875): delete failed for tablet 3b6381949cb4475490c01a7cfa8c5e20 because the tablet was not found. No further retry: Not found: Tablet not found: 3b6381949cb4475490c01a7cfa8c5e20
I20260923 18:00:40.744065 23406 ts_tablet_manager.cc:1905] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:40.746093 23406 ts_tablet_manager.cc:1918] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:40.746194 23406 log.cc:1196] T 3b6381949cb4475490c01a7cfa8c5e20 P b844678d1ab24411857fc79ae7010d2c: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/3b6381949cb4475490c01a7cfa8c5e20
I20260923 18:00:40.746326 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 23016
I20260923 18:00:40.746487 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.746145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:46199 (local address 127.0.0.1:53260)
0923 18:00:40.746325 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.746387 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:46199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260923 18:00:40.748219 22814 catalog_manager.cc:4584] TS e53e7fbff400481f8979b8602fcd6a9b (127.15.139.129:46199): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b6381949cb4475490c01a7cfa8c5e20: Network error: Client connection negotiation failed: client connection to 127.15.139.129:46199: connect: Connection refused (error 111)
I20260923 18:00:40.752557 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 23152
I20260923 18:00:40.752882 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:40.752539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:36899 (local address 127.0.0.1:59730)
0923 18:00:40.752710 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:40.752781 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.130:36899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260923 18:00:40.759817 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 22501
I20260923 18:00:40.765722 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 22605
2026-09-23T18:00:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7466 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:40 dist-test-slave-vr57 krb5kdc[23409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:40 dist-test-slave-vr57 krb5kdc[23409](info): set up 2 sockets
Sep 23 18:00:40 dist-test-slave-vr57 krb5kdc[23409](info): commencing operation
krb5kdc: starting...
W20260923 18:00:41.816524 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:41 dist-test-slave-vr57 krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:41Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.190.keytab.
Entry for principal oryx/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.190.keytab.
I20260923 18:00:41.944082 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:37677
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.190.keytab
--principal=oryx/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:37677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:42.067773 23422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:42.068189 23422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:42.068300 23422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:42.073794 23422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:42.073900 23422 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 18:00:42.073952 23422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:42.074007 23422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:42.074055 23422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:42.074100 23422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:42.080456 23422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:37677
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.190.keytab
--principal=oryx/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:37677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:42.081745 23422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:42.082909 23422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:42.091805 23427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:42.091853 23430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:42.091826 23428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:42.092397 23422 server_base.cc:1034] running on GCE node
I20260923 18:00:42.092963 23422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:42.094139 23422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:42.095320 23422 hybrid_clock.cc:648] HybridClock initialized: now 1790186442095306 us; error 30 us; skew 500 ppm
Sep 23 18:00:42 dist-test-slave-vr57 krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186442, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:42.098214 23422 init.cc:377] Logged in from keytab as oryx/127.15.139.190@KRBTEST.COM (short username oryx)
I20260923 18:00:42.099162 23422 webserver.cc:460] Webserver started at http://127.15.139.190:39047/ using document root <none> and password file <none>
I20260923 18:00:42.099838 23422 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:42.099932 23422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:42.100167 23422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:42.102278 23422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "732231d5680042a88a4f160cddcabec4"
format_stamp: "Formatted at 2026-09-23 18:00:42 on dist-test-slave-vr57"
server_key: "b2a4388f6d8516cdf9e696c3797b8759"
server_key_iv: "9de17056ea40223e6047483f93d45359"
server_key_version: "encryptionkey@0"
I20260923 18:00:42.102875 23422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "732231d5680042a88a4f160cddcabec4"
format_stamp: "Formatted at 2026-09-23 18:00:42 on dist-test-slave-vr57"
server_key: "b2a4388f6d8516cdf9e696c3797b8759"
server_key_iv: "9de17056ea40223e6047483f93d45359"
server_key_version: "encryptionkey@0"
I20260923 18:00:42.106741 23422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:00:42.109275 23437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:42.110507 23422 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:42.110675 23422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "732231d5680042a88a4f160cddcabec4"
format_stamp: "Formatted at 2026-09-23 18:00:42 on dist-test-slave-vr57"
server_key: "b2a4388f6d8516cdf9e696c3797b8759"
server_key_iv: "9de17056ea40223e6047483f93d45359"
server_key_version: "encryptionkey@0"
I20260923 18:00:42.110787 23422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:42.129916 23422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:42.133219 23422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:42.133424 23422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:42.142271 23422 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:37677
I20260923 18:00:42.142288 23499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:37677 every 8 connection(s)
I20260923 18:00:42.143256 23422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:42.146104 23500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:42.151232 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 23422
I20260923 18:00:42.151371 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:42.151698 15918 external_mini_cluster.cc:1428] Setting key 988e12a547af3ce7d3ccbce95351ad73
I20260923 18:00:42.151917 23500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4: Bootstrap starting.
I20260923 18:00:42.154402 23500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:42.155153 23500 log.cc:824] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:42.157368 23500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4: No bootstrap required, opened a new log
Sep 23 18:00:42 dist-test-slave-vr57 krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:42.161540 23500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732231d5680042a88a4f160cddcabec4" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37677 } }
I20260923 18:00:42.161767 23500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:42.161862 23500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732231d5680042a88a4f160cddcabec4, State: Initialized, Role: FOLLOWER
I20260923 18:00:42.162356 23500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732231d5680042a88a4f160cddcabec4" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37677 } }
I20260923 18:00:42.162552 23500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:42.162632 23500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:42.162744 23500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:42.163985 23500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732231d5680042a88a4f160cddcabec4" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37677 } }
I20260923 18:00:42.164399 23500 leader_election.cc:302] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 732231d5680042a88a4f160cddcabec4; no voters:
I20260923 18:00:42.164825 23500 leader_election.cc:288] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:42.164939 23505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:42.165138 23505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [term 1 LEADER]: Becoming Leader. State: Replica: 732231d5680042a88a4f160cddcabec4, State: Running, Role: LEADER
I20260923 18:00:42.165513 23505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732231d5680042a88a4f160cddcabec4" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37677 } }
I20260923 18:00:42.166172 23500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:42.167140 23506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "732231d5680042a88a4f160cddcabec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732231d5680042a88a4f160cddcabec4" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37677 } } }
I20260923 18:00:42.167251 23506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:42.167634 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.153052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50426 (local address 127.15.139.190:37677)
0923 18:00:42.153502 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:42.153517 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:42.153552 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:00:42.154192 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:42.154208 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:42.154597 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:42.154934 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:42.155825 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.157239 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.158190 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.158973 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.159139 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.161897 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:42.161906 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:42.161924 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:42.161954 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:42.163951 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.164481 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.164486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.164491 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.164666 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.165028 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.165032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.165037 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.165533 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:42.165705 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:42.166174 (+ 469us) server_negotiation.cc:299] Negotiation successful
0923 18:00:42.166472 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":143,"threads_started":1}
I20260923 18:00:42.167980 23514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:42.167702 23507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 732231d5680042a88a4f160cddcabec4. Latest consensus state: current_term: 1 leader_uuid: "732231d5680042a88a4f160cddcabec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732231d5680042a88a4f160cddcabec4" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37677 } } }
I20260923 18:00:42.168676 23507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:42.172279 23514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:42.177811 23514 catalog_manager.cc:1347] Generated new cluster ID: 937e3f4cd2d04dd0bc763eed72c44c0c
I20260923 18:00:42.177886 23514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:42.184285 23514 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:42.185173 23514 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:42.188983 23514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4: Generated new TSK 0
I20260923 18:00:42.189658 23514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab.
Entry for principal oryx/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab.
I20260923 18:00:42.222195 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:37677
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab
--principal=oryx/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:42.357155 23526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:42.357452 23526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:42.357546 23526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:42.361743 23526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:42.361809 23526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:42.361886 23526 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 18:00:42.361958 23526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:42.367581 23526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab
--principal=oryx/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.139.190:37677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:42.368700 23526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:42.369762 23526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:42.379534 23531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:42.379561 23534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:42.379561 23532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:42.380002 23526 server_base.cc:1034] running on GCE node
I20260923 18:00:42.380412 23526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:42.381078 23526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:42.382268 23526 hybrid_clock.cc:648] HybridClock initialized: now 1790186442382254 us; error 30 us; skew 500 ppm
Sep 23 18:00:42 dist-test-slave-vr57 krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186442, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:42.385293 23526 init.cc:377] Logged in from keytab as oryx/127.15.139.129@KRBTEST.COM (short username oryx)
I20260923 18:00:42.386296 23526 webserver.cc:460] Webserver started at http://127.15.139.129:45205/ using document root <none> and password file <none>
I20260923 18:00:42.386932 23526 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:42.387010 23526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:42.387277 23526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:42.389513 23526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "62fc05525542484e8baa5885fab02627"
format_stamp: "Formatted at 2026-09-23 18:00:42 on dist-test-slave-vr57"
server_key: "af422ffab3c8e6a81fff4cdc5e5e847e"
server_key_iv: "8584b3bcadecfb86604376aa181873b9"
server_key_version: "encryptionkey@0"
I20260923 18:00:42.390092 23526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "62fc05525542484e8baa5885fab02627"
format_stamp: "Formatted at 2026-09-23 18:00:42 on dist-test-slave-vr57"
server_key: "af422ffab3c8e6a81fff4cdc5e5e847e"
server_key_iv: "8584b3bcadecfb86604376aa181873b9"
server_key_version: "encryptionkey@0"
I20260923 18:00:42.394155 23526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:42.396788 23541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:42.398134 23526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:42.398265 23526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "62fc05525542484e8baa5885fab02627"
format_stamp: "Formatted at 2026-09-23 18:00:42 on dist-test-slave-vr57"
server_key: "af422ffab3c8e6a81fff4cdc5e5e847e"
server_key_iv: "8584b3bcadecfb86604376aa181873b9"
server_key_version: "encryptionkey@0"
I20260923 18:00:42.398384 23526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:42.434600 23526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:42.438124 23526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:42.438359 23526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:42.439953 23526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:42.440006 23526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:42.440070 23526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:42.440107 23526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 18:00:42 dist-test-slave-vr57 krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186442, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:42.451783 23526 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:35227
I20260923 18:00:42.451833 23656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:35227 every 8 connection(s)
I20260923 18:00:42.452996 23526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:42.455772 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.441165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:34731 (local address 127.15.139.190:37677)
0923 18:00:42.441317 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:42.441321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:42.442387 (+ 1066us) server_negotiation.cc:407] Connection header received
0923 18:00:42.443746 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:42.443749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:42.443815 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:42.443941 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:42.445330 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.446024 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.448014 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.448458 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.448586 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.451904 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:42.451908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:42.451911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:42.451946 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:42.453631 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.454229 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.454232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.454233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.454314 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.454602 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.454604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.454605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.454782 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:42.454857 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:42.455440 (+ 583us) server_negotiation.cc:299] Negotiation successful
0923 18:00:42.455651 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 18:00:42.456583 23550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.441648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:37677 (local address 127.15.139.129:34731)
0923 18:00:42.442214 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:42.442261 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:42.443401 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:42.444139 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:42.444149 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:42.444577 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:42.445165 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:42.445180 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.446175 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.446180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:42.447848 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:42.447864 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.448630 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.448641 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:42.448867 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.449645 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:42.449670 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:42.451724 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:42.453780 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:42.453786 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:42.453801 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:42.454111 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:42.454379 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:42.454381 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:42.454383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:42.454509 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:42.454875 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:42.454880 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:42.455169 (+ 289us) client_negotiation.cc:769] Sending connection context
0923 18:00:42.455331 (+ 162us) client_negotiation.cc:240] Negotiation successful
0923 18:00:42.455672 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":155,"threads_started":1}
I20260923 18:00:42.457990 23657 heartbeater.cc:347] Connected to a master server at 127.15.139.190:37677
I20260923 18:00:42.458225 23657 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:42.458740 23657 heartbeater.cc:511] Master 127.15.139.190:37677 requested a full tablet report, sending...
I20260923 18:00:42.460194 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 23526
I20260923 18:00:42.460318 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:42.460314 23454 ts_manager.cc:194] Registered new tserver with Master: 62fc05525542484e8baa5885fab02627 (127.15.139.129:35227)
I20260923 18:00:42.460654 15918 external_mini_cluster.cc:1428] Setting key 856805d099e2cc8235d566f67474ae54
I20260923 18:00:42.461995 23454 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:34731
I20260923 18:00:42.468838 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.460374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:45405 (local address 127.15.139.190:37677)
0923 18:00:42.460494 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:42.460499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:42.460813 (+ 314us) server_negotiation.cc:407] Connection header received
0923 18:00:42.460932 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:42.460935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:42.460975 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:42.461049 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:42.461611 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.462326 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.463455 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.463780 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.463856 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.465275 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:42.465279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:42.465282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:42.465308 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:42.466824 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.467441 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.467446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.467448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.467592 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.467935 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.467940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.467942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.468138 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:42.468255 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:42.468545 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 18:00:42.468661 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 18:00:42.468856 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.460464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:37677 (local address 127.15.139.129:45405)
0923 18:00:42.460720 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:42.460734 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:42.460850 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:42.461154 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:42.461157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:42.461358 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:42.461474 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:42.461481 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.462449 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.462452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:42.463298 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:42.463308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.464021 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.464025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:42.464110 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.464681 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:42.464694 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:42.465105 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:42.467009 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:42.467020 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:42.467022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:42.467331 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:42.467702 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:42.467707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:42.467710 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:42.467814 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:42.468245 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:42.468248 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:42.468356 (+ 108us) client_negotiation.cc:769] Sending connection context
0923 18:00:42.468475 (+ 119us) client_negotiation.cc:240] Negotiation successful
0923 18:00:42.468632 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for oryx/127.15.139.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab.
Entry for principal oryx/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab.
I20260923 18:00:42.489351 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:37677
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab
--principal=oryx/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:42.602661 23666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:42.602932 23666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:42.602977 23666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:42.607650 23666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:42.607707 23666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:42.607749 23666 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 18:00:42.607825 23666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:42.613343 23666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab
--principal=oryx/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.139.190:37677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:42.614534 23666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:42.615800 23666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:42.625813 23671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:42.625806 23672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:42.625857 23674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:42.626205 23666 server_base.cc:1034] running on GCE node
I20260923 18:00:42.626705 23666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:42.627458 23666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:42.628724 23666 hybrid_clock.cc:648] HybridClock initialized: now 1790186442628717 us; error 43 us; skew 500 ppm
Sep 23 18:00:42 dist-test-slave-vr57 krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186442, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:42.631752 23666 init.cc:377] Logged in from keytab as oryx/127.15.139.130@KRBTEST.COM (short username oryx)
I20260923 18:00:42.632776 23666 webserver.cc:460] Webserver started at http://127.15.139.130:34509/ using document root <none> and password file <none>
I20260923 18:00:42.633351 23666 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:42.633412 23666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:42.633580 23666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:42.635692 23666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9"
format_stamp: "Formatted at 2026-09-23 18:00:42 on dist-test-slave-vr57"
server_key: "fe6c00661d7a83abbf4fc1dec3b45451"
server_key_iv: "bac12f2e96c35714292a4ef5f0fdee70"
server_key_version: "encryptionkey@0"
I20260923 18:00:42.636241 23666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9"
format_stamp: "Formatted at 2026-09-23 18:00:42 on dist-test-slave-vr57"
server_key: "fe6c00661d7a83abbf4fc1dec3b45451"
server_key_iv: "bac12f2e96c35714292a4ef5f0fdee70"
server_key_version: "encryptionkey@0"
I20260923 18:00:42.640054 23666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260923 18:00:42.642444 23681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:42.643959 23666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:00:42.644116 23666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9"
format_stamp: "Formatted at 2026-09-23 18:00:42 on dist-test-slave-vr57"
server_key: "fe6c00661d7a83abbf4fc1dec3b45451"
server_key_iv: "bac12f2e96c35714292a4ef5f0fdee70"
server_key_version: "encryptionkey@0"
I20260923 18:00:42.644265 23666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:42.687796 23666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:42.691380 23666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:42.691656 23666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:42.693213 23666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:42.693293 23666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:42.693373 23666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:42.693434 23666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 18:00:42 dist-test-slave-vr57 krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186442, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:42.704823 23666 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:37837
I20260923 18:00:42.705164 23796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:37837 every 8 connection(s)
I20260923 18:00:42.705823 23666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:42.707095 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 23666
I20260923 18:00:42.707212 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:42.707645 15918 external_mini_cluster.cc:1428] Setting key d4462a4c3750a9819565ebf4e99e7e7b
I20260923 18:00:42.708715 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.694032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:47451 (local address 127.15.139.190:37677)
0923 18:00:42.694200 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:42.694209 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:42.695113 (+ 904us) server_negotiation.cc:407] Connection header received
0923 18:00:42.696151 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:42.696155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:42.696213 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:42.696284 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:42.697893 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.698774 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.700702 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.701106 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.701233 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.704481 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:42.704485 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:42.704488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:42.704524 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:42.706205 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.706869 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.706874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.706875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.706970 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.707432 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.707436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.707439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.707716 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:42.707816 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:42.708387 (+ 571us) server_negotiation.cc:299] Negotiation successful
0923 18:00:42.708539 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 18:00:42.709581 23690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.694463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:37677 (local address 127.15.139.130:47451)
0923 18:00:42.694941 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:42.694992 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:42.695928 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:42.696514 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:42.696525 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:42.697034 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:42.697737 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:42.697757 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.698947 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.698954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:42.700556 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:42.700572 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.701296 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.701304 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:42.701548 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.702216 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:42.702243 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:42.704300 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:42.706336 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:42.706342 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:42.706358 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:42.706755 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:42.707088 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:42.707091 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:42.707092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:42.707307 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:42.707832 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:42.707837 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:42.708132 (+ 295us) client_negotiation.cc:769] Sending connection context
0923 18:00:42.708323 (+ 191us) client_negotiation.cc:240] Negotiation successful
0923 18:00:42.708665 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":137,"threads_started":1}
I20260923 18:00:42.711014 23797 heartbeater.cc:347] Connected to a master server at 127.15.139.190:37677
I20260923 18:00:42.711162 23797 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:42.711796 23797 heartbeater.cc:511] Master 127.15.139.190:37677 requested a full tablet report, sending...
I20260923 18:00:42.712961 23453 ts_manager.cc:194] Registered new tserver with Master: 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837)
I20260923 18:00:42.713833 23453 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:47451
WARNING: no policy specified for oryx/127.15.139.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.131@KRBTEST.COM" created.
I20260923 18:00:42.722236 23804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.713278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:37677 (local address 127.15.139.130:50199)
0923 18:00:42.714322 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:42.714339 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:42.714450 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:42.714715 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:42.714720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:42.714983 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:42.715154 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:42.715165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.716385 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.716390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:42.717355 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:42.717367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.717838 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.717845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:42.717971 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.718566 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:42.718580 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:42.718967 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:42.720639 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:42.720645 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:42.720647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:42.720843 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:42.721131 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:42.721133 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:42.721135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:42.721220 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:42.721748 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:42.721751 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:42.721830 (+ 79us) client_negotiation.cc:769] Sending connection context
0923 18:00:42.721923 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 18:00:42.722096 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"thread_start_us":102,"threads_started":1}
I20260923 18:00:42.722374 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.713335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:50199 (local address 127.15.139.190:37677)
0923 18:00:42.713474 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:42.713480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:42.714419 (+ 939us) server_negotiation.cc:407] Connection header received
0923 18:00:42.714540 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:42.714543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:42.714591 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:42.714678 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:42.715312 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.716218 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.717503 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.717712 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.717788 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.719105 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:42.719108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:42.719110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:42.719131 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:42.720507 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.720945 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.720949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.720951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.721033 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.721319 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.721324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.721326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.721634 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:42.721740 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:42.722010 (+ 270us) server_negotiation.cc:299] Negotiation successful
0923 18:00:42.722139 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab.
Entry for principal oryx/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab.
I20260923 18:00:42.736594 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:37677
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab
--principal=oryx/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:42.849663 23806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:42.849923 23806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:42.849967 23806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:42.854190 23806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:42.854265 23806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:42.854305 23806 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 18:00:42.854367 23806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:42.860138 23806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab
--principal=oryx/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.139.190:37677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:42.861214 23806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:42.862296 23806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:42.872133 23812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:42.872189 23811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:42.872140 23814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:42.872330 23806 server_base.cc:1034] running on GCE node
I20260923 18:00:42.872851 23806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:42.873548 23806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:42.874740 23806 hybrid_clock.cc:648] HybridClock initialized: now 1790186442874720 us; error 29 us; skew 500 ppm
Sep 23 18:00:42 dist-test-slave-vr57 krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186442, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:42.877876 23806 init.cc:377] Logged in from keytab as oryx/127.15.139.131@KRBTEST.COM (short username oryx)
I20260923 18:00:42.878847 23806 webserver.cc:460] Webserver started at http://127.15.139.131:33755/ using document root <none> and password file <none>
I20260923 18:00:42.879420 23806 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:42.879541 23806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:42.879807 23806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:42.881880 23806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "93fe794d6c76416abe4b47bf7ef90b3d"
format_stamp: "Formatted at 2026-09-23 18:00:42 on dist-test-slave-vr57"
server_key: "95bc0d4835c408a1c18e3abfdf9e5b40"
server_key_iv: "f98dea1b42481e8df3d25a5640f40e8b"
server_key_version: "encryptionkey@0"
I20260923 18:00:42.882542 23806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "93fe794d6c76416abe4b47bf7ef90b3d"
format_stamp: "Formatted at 2026-09-23 18:00:42 on dist-test-slave-vr57"
server_key: "95bc0d4835c408a1c18e3abfdf9e5b40"
server_key_iv: "f98dea1b42481e8df3d25a5640f40e8b"
server_key_version: "encryptionkey@0"
I20260923 18:00:42.886476 23806 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260923 18:00:42.889053 23821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:42.890477 23806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260923 18:00:42.890615 23806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "93fe794d6c76416abe4b47bf7ef90b3d"
format_stamp: "Formatted at 2026-09-23 18:00:42 on dist-test-slave-vr57"
server_key: "95bc0d4835c408a1c18e3abfdf9e5b40"
server_key_iv: "f98dea1b42481e8df3d25a5640f40e8b"
server_key_version: "encryptionkey@0"
I20260923 18:00:42.890744 23806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:42.928763 23806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:42.932291 23806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:42.932510 23806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:42.934041 23806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:42.934123 23806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:42.934199 23806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:42.934244 23806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 18:00:42 dist-test-slave-vr57 krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186442, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:42.945319 23806 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:32975
I20260923 18:00:42.945629 23936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:32975 every 8 connection(s)
I20260923 18:00:42.946591 23806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:42.949059 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.934884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:48507 (local address 127.15.139.190:37677)
0923 18:00:42.935040 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:42.935045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:42.936128 (+ 1083us) server_negotiation.cc:407] Connection header received
0923 18:00:42.937500 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:42.937504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:42.937576 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:42.937680 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:42.939539 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.940200 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.941506 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.941737 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.941853 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.944664 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:42.944670 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:42.944673 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:42.944703 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:42.946777 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.947409 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.947412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.947414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.947582 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.947938 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.947940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.947941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.948198 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:42.948339 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:42.948832 (+ 493us) server_negotiation.cc:299] Negotiation successful
0923 18:00:42.948938 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 18:00:42.949864 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.935357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:37677 (local address 127.15.139.131:48507)
0923 18:00:42.935944 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:42.936003 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:42.937213 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:42.937951 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:42.937981 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:42.938502 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:42.939318 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:42.939343 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.940341 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.940345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:42.941370 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:42.941380 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.941906 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.941911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:42.942085 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.942728 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:42.942744 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:42.944474 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:42.946893 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:42.946898 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:42.946910 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:42.947284 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:42.947673 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:42.947676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:42.947678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:42.947845 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:42.948336 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:42.948341 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:42.948586 (+ 245us) client_negotiation.cc:769] Sending connection context
0923 18:00:42.948747 (+ 161us) client_negotiation.cc:240] Negotiation successful
0923 18:00:42.949000 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":186,"threads_started":1}
I20260923 18:00:42.951159 23937 heartbeater.cc:347] Connected to a master server at 127.15.139.190:37677
I20260923 18:00:42.951318 23937 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:42.951965 23937 heartbeater.cc:511] Master 127.15.139.190:37677 requested a full tablet report, sending...
I20260923 18:00:42.953090 23453 ts_manager.cc:194] Registered new tserver with Master: 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131:32975)
I20260923 18:00:42.953788 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 23806
I20260923 18:00:42.953917 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:00:42.953893 23453 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:48507
I20260923 18:00:42.954335 15918 external_mini_cluster.cc:1428] Setting key bf9627621fee228beba41095f5b4716a
I20260923 18:00:42.956491 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:00:42.961877 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.953229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:49639 (local address 127.15.139.190:37677)
0923 18:00:42.953350 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:42.953353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:42.953611 (+ 258us) server_negotiation.cc:407] Connection header received
0923 18:00:42.953727 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:42.953731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:42.953782 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:42.953864 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:42.954428 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.955116 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.956155 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.956382 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.956492 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.957998 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:42.958004 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:42.958010 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:42.958042 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:42.960133 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.960650 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.960653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.960654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.960734 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:42.961037 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:42.961040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:42.961041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:42.961228 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:42.961310 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:42.961613 (+ 303us) server_negotiation.cc:299] Negotiation successful
0923 18:00:42.961722 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 18:00:42.962005 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.953165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:37677 (local address 127.15.139.131:49639)
0923 18:00:42.953523 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:42.953538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:42.953631 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:42.953892 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:42.953895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:42.954105 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:42.954312 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:42.954320 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.955260 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.955263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:42.956025 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:42.956034 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.956496 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.956499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:42.956664 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.957488 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:42.957501 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:42.957856 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:42.960250 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:42.960252 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:42.960254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:42.960538 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:42.960829 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:42.960835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:42.960840 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:42.960942 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:42.961327 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:42.961333 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:42.961446 (+ 113us) client_negotiation.cc:769] Sending connection context
0923 18:00:42.961576 (+ 130us) client_negotiation.cc:240] Negotiation successful
0923 18:00:42.961748 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":88,"threads_started":1}
Sep 23 18:00:42 dist-test-slave-vr57 krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186442, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 18:00:42 dist-test-slave-vr57 krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:42 dist-test-slave-vr57 krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 18:00:42.986377 23503 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:42.977631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50432 (local address 127.15.139.190:37677)
0923 18:00:42.977793 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:42.977796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:42.977810 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:00:42.977854 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:42.977856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:42.977896 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:42.977969 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:42.979065 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.980063 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.981485 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.981841 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.981961 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.986217 (+ 4256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50432: BlockingRecv error: recv got EOF from 127.0.0.1:50432 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 23 18:00:42 dist-test-slave-vr57 krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186442, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:43.001956 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:42.993198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50444 (local address 127.15.139.190:37677)
0923 18:00:42.993320 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:42.993325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:42.993508 (+ 183us) server_negotiation.cc:407] Connection header received
0923 18:00:42.993567 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:42.993569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:42.993614 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:42.993700 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:42.994279 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.994905 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.995811 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:42.996102 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:42.996189 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:42.998693 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:42.998696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:42.998699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:42.998724 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:43.000220 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:43.000661 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:43.000665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:43.000667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:43.000752 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:43.001048 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:43.001052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:43.001053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:43.001241 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:43.001384 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.001661 (+ 277us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.001776 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 18:00:43.004907 23453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:00:43.008601 23453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:43.023758 23959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.017821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35227 (local address 127.0.0.1:37758)
0923 18:00:43.018210 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.018267 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.018425 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.019434 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.019440 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.019466 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:43.019684 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.019695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.021120 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.021123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.022158 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.022171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.023185 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.023189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.023295 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.023328 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.023459 (+ 131us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.023605 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":52,"thread_start_us":91,"threads_started":1}
I20260923 18:00:43.023758 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.018538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37758 (local address 127.15.139.129:35227)
0923 18:00:43.018936 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.018944 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.018971 (+ 27us) server_negotiation.cc:407] Connection header received
0923 18:00:43.019058 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.019064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.019259 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.019436 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:43.019841 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.020987 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.022305 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.023056 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.023187 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.023325 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.023490 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.023645 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":82,"threads_started":1}
I20260923 18:00:43.024034 23960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.017907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:37837 (local address 127.0.0.1:48618)
0923 18:00:43.018538 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.018553 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.018648 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.019810 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.019816 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.019838 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:43.019960 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.019967 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.021240 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.021244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.022450 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.022462 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.023632 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.023636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.023725 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.023753 (+ 28us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.023806 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.023875 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":21,"thread_start_us":80,"threads_started":1}
I20260923 18:00:43.025198 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.019035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48618 (local address 127.15.139.130:37837)
0923 18:00:43.019310 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.019321 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.019342 (+ 21us) server_negotiation.cc:407] Connection header received
0923 18:00:43.019415 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.019423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.019651 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.020124 (+ 473us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:43.020258 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.021124 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.022603 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.023441 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.024750 (+ 1309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.024889 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.024939 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.025041 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":113,"threads_started":1}
I20260923 18:00:43.025275 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.018269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:32975 (local address 127.0.0.1:49092)
0923 18:00:43.019804 (+ 1535us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.019816 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.020056 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.020567 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.020570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.020588 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:43.020659 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.020664 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.022030 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.022034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.023449 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.023459 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.024836 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.024840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.024950 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.024975 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.025059 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.025135 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1460,"thread_start_us":114,"threads_started":1}
I20260923 18:00:43.025349 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.018730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49092 (local address 127.15.139.131:32975)
0923 18:00:43.019114 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.019129 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.019888 (+ 759us) server_negotiation.cc:407] Connection header received
0923 18:00:43.020220 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.020227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.020405 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.020564 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:43.020751 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.021871 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.023619 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.024475 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.024597 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.024734 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.025112 (+ 378us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.025214 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":137,"threads_started":1}
I20260923 18:00:43.026865 23591 tablet_service.cc:1467] Processing CreateTablet for tablet c130752ca22546a28473d9e7c8f6ba2c (DEFAULT_TABLE table=test-table [id=684b503f2ce14e8f8e9f6e58442c38e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:43.027755 23871 tablet_service.cc:1467] Processing CreateTablet for tablet c130752ca22546a28473d9e7c8f6ba2c (DEFAULT_TABLE table=test-table [id=684b503f2ce14e8f8e9f6e58442c38e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:43.028036 23591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c130752ca22546a28473d9e7c8f6ba2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.028177 23731 tablet_service.cc:1467] Processing CreateTablet for tablet c130752ca22546a28473d9e7c8f6ba2c (DEFAULT_TABLE table=test-table [id=684b503f2ce14e8f8e9f6e58442c38e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:43.028647 23871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c130752ca22546a28473d9e7c8f6ba2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.029090 23731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c130752ca22546a28473d9e7c8f6ba2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.035351 23965 tablet_bootstrap.cc:492] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d: Bootstrap starting.
I20260923 18:00:43.036768 23966 tablet_bootstrap.cc:492] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627: Bootstrap starting.
I20260923 18:00:43.036826 23967 tablet_bootstrap.cc:492] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9: Bootstrap starting.
I20260923 18:00:43.038189 23965 tablet_bootstrap.cc:654] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.038902 23967 tablet_bootstrap.cc:654] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.038918 23966 tablet_bootstrap.cc:654] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.039023 23965 log.cc:824] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:43.039799 23966 log.cc:824] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:43.039798 23967 log.cc:824] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:43.041211 23965 tablet_bootstrap.cc:492] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d: No bootstrap required, opened a new log
I20260923 18:00:43.041433 23965 ts_tablet_manager.cc:1397] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260923 18:00:43.041682 23967 tablet_bootstrap.cc:492] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9: No bootstrap required, opened a new log
I20260923 18:00:43.041865 23967 ts_tablet_manager.cc:1397] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260923 18:00:43.042039 23966 tablet_bootstrap.cc:492] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627: No bootstrap required, opened a new log
I20260923 18:00:43.042220 23966 ts_tablet_manager.cc:1397] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260923 18:00:43.045084 23965 raft_consensus.cc:348] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.045352 23965 raft_consensus.cc:374] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.045413 23965 raft_consensus.cc:729] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fe794d6c76416abe4b47bf7ef90b3d, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.045873 23965 consensus_queue.cc:260] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.046031 23967 raft_consensus.cc:348] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.046325 23967 raft_consensus.cc:374] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.046382 23967 raft_consensus.cc:729] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3617e62d4a45dc9a021fe4a1eff1f9, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.046885 23967 consensus_queue.cc:260] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.046958 23966 raft_consensus.cc:348] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.047204 23937 heartbeater.cc:503] Master 127.15.139.190:37677 was elected leader, sending a full tablet report...
I20260923 18:00:43.047245 23966 raft_consensus.cc:374] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.047307 23966 raft_consensus.cc:729] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fc05525542484e8baa5885fab02627, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.047766 23966 consensus_queue.cc:260] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.047925 23967 ts_tablet_manager.cc:1428] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260923 18:00:43.048370 23797 heartbeater.cc:503] Master 127.15.139.190:37677 was elected leader, sending a full tablet report...
I20260923 18:00:43.048653 23966 ts_tablet_manager.cc:1428] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260923 18:00:43.049031 23657 heartbeater.cc:503] Master 127.15.139.190:37677 was elected leader, sending a full tablet report...
I20260923 18:00:43.049855 23965 ts_tablet_manager.cc:1428] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260923 18:00:43.158358 23971 raft_consensus.cc:482] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:43.158634 23971 raft_consensus.cc:504] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.159983 23971 leader_election.cc:288] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62fc05525542484e8baa5885fab02627 (127.15.139.129:35227), 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837)
I20260923 18:00:43.166488 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.160363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:37837 (local address 127.15.139.131:44543)
0923 18:00:43.160795 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.160812 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.160910 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.161260 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.161264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.161282 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:43.161418 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.161425 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.162765 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.162769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.164873 (+ 2104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.164886 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.166003 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.166007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.166138 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.166165 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.166249 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.166322 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":118,"threads_started":1}
I20260923 18:00:43.166669 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.160489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:44543 (local address 127.15.139.130:37837)
0923 18:00:43.160607 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.160613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.160900 (+ 287us) server_negotiation.cc:407] Connection header received
0923 18:00:43.161086 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.161090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.161147 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.161231 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:43.161612 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.162611 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.165089 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.165738 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.165874 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.165934 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.166300 (+ 366us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.166455 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:00:43.167529 23751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c130752ca22546a28473d9e7c8f6ba2c" candidate_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" is_pre_election: true
I20260923 18:00:43.167901 23751 raft_consensus.cc:2454] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93fe794d6c76416abe4b47bf7ef90b3d in term 0.
I20260923 18:00:43.168519 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.160284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:48927 (local address 127.15.139.129:35227)
0923 18:00:43.160437 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.160444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.160499 (+ 55us) server_negotiation.cc:407] Connection header received
0923 18:00:43.160717 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.160721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.160780 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.160883 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:43.161183 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.165258 (+ 4075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.167116 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.167808 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.167945 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.168002 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.168295 (+ 293us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.168398 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 18:00:43.168572 23825 leader_election.cc:302] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3617e62d4a45dc9a021fe4a1eff1f9, 93fe794d6c76416abe4b47bf7ef90b3d; no voters:
I20260923 18:00:43.168861 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.160191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35227 (local address 127.15.139.131:48927)
0923 18:00:43.160415 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.160442 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.160580 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.160923 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.160927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.160947 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:43.161066 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.161073 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.165394 (+ 4321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.165398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.166748 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.166759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.167969 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.167975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.168129 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.168162 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.168262 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.168346 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260923 18:00:43.168883 23971 raft_consensus.cc:2781] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:43.168972 23971 raft_consensus.cc:482] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:43.169029 23971 raft_consensus.cc:3037] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.170179 23971 raft_consensus.cc:504] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.170619 23971 leader_election.cc:288] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [CANDIDATE]: Term 1 election: Requested vote from peers 62fc05525542484e8baa5885fab02627 (127.15.139.129:35227), 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837)
I20260923 18:00:43.171044 23593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c130752ca22546a28473d9e7c8f6ba2c" candidate_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc05525542484e8baa5885fab02627" is_pre_election: true
I20260923 18:00:43.171044 23611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c130752ca22546a28473d9e7c8f6ba2c" candidate_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc05525542484e8baa5885fab02627"
I20260923 18:00:43.171392 23593 raft_consensus.cc:2454] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93fe794d6c76416abe4b47bf7ef90b3d in term 0.
I20260923 18:00:43.172030 23751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c130752ca22546a28473d9e7c8f6ba2c" candidate_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9"
I20260923 18:00:43.172171 23751 raft_consensus.cc:3037] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.173453 23751 raft_consensus.cc:2454] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93fe794d6c76416abe4b47bf7ef90b3d in term 1.
I20260923 18:00:43.173779 23825 leader_election.cc:302] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e3617e62d4a45dc9a021fe4a1eff1f9, 93fe794d6c76416abe4b47bf7ef90b3d; no voters: 62fc05525542484e8baa5885fab02627
I20260923 18:00:43.173962 23971 raft_consensus.cc:2781] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:43.174631 23971 raft_consensus.cc:686] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 LEADER]: Becoming Leader. State: Replica: 93fe794d6c76416abe4b47bf7ef90b3d, State: Running, Role: LEADER
I20260923 18:00:43.175238 23971 consensus_queue.cc:237] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.179544 23453 catalog_manager.cc:5495] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131). New cstate: current_term: 1 leader_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } health_report { overall_health: HEALTHY } } }
W20260923 18:00:43.197643 23938 tablet.cc:2369] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:00:43.204474 23658 tablet.cc:2369] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:00:43.208787 23798 tablet.cc:2369] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:00:43.223322 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.219723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.15.139.131:32975)
0923 18:00:43.219871 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.219876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.219891 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:00:43.220056 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.220059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.220124 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.220237 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:43.220470 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.220995 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.221846 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.222090 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.222202 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.222483 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:43.222922 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:43.223052 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.223116 (+ 64us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.223173 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 18:00:43.229183 23611 raft_consensus.cc:3037] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.229393 23751 raft_consensus.cc:1261] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Refusing update from remote peer 93fe794d6c76416abe4b47bf7ef90b3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:43.230082 23971 consensus_queue.cc:1035] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:43.230679 23611 raft_consensus.cc:1261] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Refusing update from remote peer 93fe794d6c76416abe4b47bf7ef90b3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:43.231240 23971 consensus_queue.cc:1035] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:43.238315 23979 mvcc.cc:205] Tried to move back new op lower bound from 7332603671459450880 to 7332603671249969152. Current Snapshot: MvccSnapshot[applied={T|T < 7332603671459450880}]
I20260923 18:00:43.239636 23978 mvcc.cc:205] Tried to move back new op lower bound from 7332603671459450880 to 7332603671249969152. Current Snapshot: MvccSnapshot[applied={T|T < 7332603671459450880}]
I20260923 18:00:43.238982 23980 mvcc.cc:205] Tried to move back new op lower bound from 7332603671459450880 to 7332603671249969152. Current Snapshot: MvccSnapshot[applied={T|T < 7332603671459450880}]
I20260923 18:00:43.247722 23451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:43.247908 23451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:43.250500 23451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4: Sending DeleteTablet for 3 replicas of tablet c130752ca22546a28473d9e7c8f6ba2c
I20260923 18:00:43.251431 23591 tablet_service.cc:1514] Processing DeleteTablet for tablet c130752ca22546a28473d9e7c8f6ba2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:37758
I20260923 18:00:43.251585 23871 tablet_service.cc:1514] Processing DeleteTablet for tablet c130752ca22546a28473d9e7c8f6ba2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:49092
I20260923 18:00:43.251794 23731 tablet_service.cc:1514] Processing DeleteTablet for tablet c130752ca22546a28473d9e7c8f6ba2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:48618
I20260923 18:00:43.252068 23988 tablet_replica.cc:330] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627: stopping tablet replica
I20260923 18:00:43.252381 23990 tablet_replica.cc:330] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9: stopping tablet replica
I20260923 18:00:43.252616 23988 raft_consensus.cc:2229] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:43.252797 23990 raft_consensus.cc:2229] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:43.252885 23451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 18:00:43.253161 23988 raft_consensus.cc:2258] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:43.253316 23990 raft_consensus.cc:2258] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 18:00:43.253340 23451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:43.254765 23988 ts_tablet_manager.cc:1905] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:43.254976 23990 ts_tablet_manager.cc:1905] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:43.257730 23988 ts_tablet_manager.cc:1918] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:43.257874 23988 log.cc:1196] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/c130752ca22546a28473d9e7c8f6ba2c
I20260923 18:00:43.258018 23990 ts_tablet_manager.cc:1918] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:43.258183 23990 log.cc:1196] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/c130752ca22546a28473d9e7c8f6ba2c
I20260923 18:00:43.258600 23990 ts_tablet_manager.cc:1939] T c130752ca22546a28473d9e7c8f6ba2c P 0e3617e62d4a45dc9a021fe4a1eff1f9: Deleting consensus metadata
I20260923 18:00:43.258697 23591 tablet_service.cc:1467] Processing CreateTablet for tablet 267822f27de64864b4d45634cd6a9691 (DEFAULT_TABLE table=test-table [id=0b6c66f62d534e3a87844e2571c2844c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:43.258980 23591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267822f27de64864b4d45634cd6a9691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.259083 23731 tablet_service.cc:1467] Processing CreateTablet for tablet 267822f27de64864b4d45634cd6a9691 (DEFAULT_TABLE table=test-table [id=0b6c66f62d534e3a87844e2571c2844c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:43.259115 23871 tablet_service.cc:1467] Processing CreateTablet for tablet 267822f27de64864b4d45634cd6a9691 (DEFAULT_TABLE table=test-table [id=0b6c66f62d534e3a87844e2571c2844c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:43.259380 23871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267822f27de64864b4d45634cd6a9691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.259473 23731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267822f27de64864b4d45634cd6a9691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.258277 23988 ts_tablet_manager.cc:1939] T c130752ca22546a28473d9e7c8f6ba2c P 62fc05525542484e8baa5885fab02627: Deleting consensus metadata
I20260923 18:00:43.261768 23965 tablet_bootstrap.cc:492] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d: Bootstrap starting.
I20260923 18:00:43.262096 23967 tablet_bootstrap.cc:492] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Bootstrap starting.
I20260923 18:00:43.262166 23966 tablet_bootstrap.cc:492] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627: Bootstrap starting.
I20260923 18:00:43.262789 23965 tablet_bootstrap.cc:654] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.263108 23967 tablet_bootstrap.cc:654] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.263130 23966 tablet_bootstrap.cc:654] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.264472 23441 catalog_manager.cc:4841] TS 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837): tablet c130752ca22546a28473d9e7c8f6ba2c (table test-table [id=684b503f2ce14e8f8e9f6e58442c38e7]) successfully deleted
I20260923 18:00:43.264472 23439 catalog_manager.cc:4841] TS 62fc05525542484e8baa5885fab02627 (127.15.139.129:35227): tablet c130752ca22546a28473d9e7c8f6ba2c (table test-table [id=684b503f2ce14e8f8e9f6e58442c38e7]) successfully deleted
I20260923 18:00:43.264941 23966 tablet_bootstrap.cc:492] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627: No bootstrap required, opened a new log
I20260923 18:00:43.265033 23966 ts_tablet_manager.cc:1397] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:43.265177 23989 tablet_replica.cc:330] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d: stopping tablet replica
I20260923 18:00:43.265542 23989 raft_consensus.cc:2229] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:43.265522 23966 raft_consensus.cc:348] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.265645 23966 raft_consensus.cc:374] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.265679 23966 raft_consensus.cc:729] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fc05525542484e8baa5885fab02627, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.265841 23966 consensus_queue.cc:260] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.265959 23989 raft_consensus.cc:2258] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:43.265978 23973 raft_consensus.cc:482] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:43.266076 23973 raft_consensus.cc:504] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.266731 23966 ts_tablet_manager.cc:1428] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:43.267235 23973 leader_election.cc:288] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837), 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131:32975)
I20260923 18:00:43.268885 23989 ts_tablet_manager.cc:1905] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:43.270995 23965 tablet_bootstrap.cc:492] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d: No bootstrap required, opened a new log
I20260923 18:00:43.271090 23965 ts_tablet_manager.cc:1397] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260923 18:00:43.271627 23965 raft_consensus.cc:348] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.271752 23965 raft_consensus.cc:374] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.271793 23965 raft_consensus.cc:729] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fe794d6c76416abe4b47bf7ef90b3d, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.271948 23965 consensus_queue.cc:260] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.272217 23965 ts_tablet_manager.cc:1428] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:43.272637 23989 ts_tablet_manager.cc:1918] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:43.272740 23989 log.cc:1196] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/c130752ca22546a28473d9e7c8f6ba2c
I20260923 18:00:43.273068 23989 ts_tablet_manager.cc:1939] T c130752ca22546a28473d9e7c8f6ba2c P 93fe794d6c76416abe4b47bf7ef90b3d: Deleting consensus metadata
I20260923 18:00:43.274232 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.267674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:35229 (local address 127.15.139.130:37837)
0923 18:00:43.267788 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.267793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.269262 (+ 1469us) server_negotiation.cc:407] Connection header received
0923 18:00:43.269470 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.269473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.269522 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.269570 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:43.270002 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.270735 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.272857 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.273472 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.273561 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.273624 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.274014 (+ 390us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.274097 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 18:00:43.274220 23438 catalog_manager.cc:4841] TS 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131:32975): tablet c130752ca22546a28473d9e7c8f6ba2c (table test-table [id=684b503f2ce14e8f8e9f6e58442c38e7]) successfully deleted
I20260923 18:00:43.274787 23996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.267458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:37837 (local address 127.15.139.129:35229)
0923 18:00:43.269178 (+ 1720us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.269194 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.269337 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.269720 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.269724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.269743 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:43.269862 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.269867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.270876 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.270879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.272613 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.272623 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.273774 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.273778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.273870 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.273892 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.274540 (+ 648us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.274637 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1651,"thread_start_us":417,"threads_started":1}
I20260923 18:00:43.275352 23751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267822f27de64864b4d45634cd6a9691" candidate_uuid: "62fc05525542484e8baa5885fab02627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" is_pre_election: true
W20260923 18:00:43.275933 23545 leader_election.cc:341] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837): Illegal state: must be running to vote when last-logged opid is not known
I20260923 18:00:43.276963 23967 tablet_bootstrap.cc:492] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9: No bootstrap required, opened a new log
I20260923 18:00:43.277108 23967 ts_tablet_manager.cc:1397] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260923 18:00:43.277570 23967 raft_consensus.cc:348] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.277700 23967 raft_consensus.cc:374] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.277745 23967 raft_consensus.cc:729] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3617e62d4a45dc9a021fe4a1eff1f9, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.277902 23967 consensus_queue.cc:260] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.278301 23967 ts_tablet_manager.cc:1428] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260923 18:00:43.279448 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.268182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:51937 (local address 127.15.139.131:32975)
0923 18:00:43.268283 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.268287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.275152 (+ 6865us) server_negotiation.cc:407] Connection header received
0923 18:00:43.275335 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.275338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.275382 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.275445 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:43.275881 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.276501 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.278180 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.278834 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.278962 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.279016 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.279245 (+ 229us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.279321 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 18:00:43.279881 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.267892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:32975 (local address 127.15.139.129:51937)
0923 18:00:43.275065 (+ 7173us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.275082 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.275217 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.275600 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.275603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.275620 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:43.275733 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.275740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.276634 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.276638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.277980 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.277991 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.278970 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.278974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.279089 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.279112 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.279622 (+ 510us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.279724 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7102,"thread_start_us":79,"threads_started":1}
I20260923 18:00:43.280773 23891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267822f27de64864b4d45634cd6a9691" candidate_uuid: "62fc05525542484e8baa5885fab02627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" is_pre_election: true
I20260923 18:00:43.281013 23891 raft_consensus.cc:2454] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62fc05525542484e8baa5885fab02627 in term 0.
I20260923 18:00:43.281420 23542 leader_election.cc:302] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62fc05525542484e8baa5885fab02627, 93fe794d6c76416abe4b47bf7ef90b3d; no voters: 0e3617e62d4a45dc9a021fe4a1eff1f9
I20260923 18:00:43.281736 23994 raft_consensus.cc:2781] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:43.281817 23994 raft_consensus.cc:482] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:43.281870 23994 raft_consensus.cc:3037] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.283131 23994 raft_consensus.cc:504] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.283651 23994 leader_election.cc:288] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [CANDIDATE]: Term 1 election: Requested vote from peers 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837), 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131:32975)
I20260923 18:00:43.283959 23751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267822f27de64864b4d45634cd6a9691" candidate_uuid: "62fc05525542484e8baa5885fab02627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9"
I20260923 18:00:43.284082 23751 raft_consensus.cc:3037] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.284049 23891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267822f27de64864b4d45634cd6a9691" candidate_uuid: "62fc05525542484e8baa5885fab02627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fe794d6c76416abe4b47bf7ef90b3d"
I20260923 18:00:43.284163 23891 raft_consensus.cc:3037] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.285055 23751 raft_consensus.cc:2454] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62fc05525542484e8baa5885fab02627 in term 1.
I20260923 18:00:43.285336 23545 leader_election.cc:302] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3617e62d4a45dc9a021fe4a1eff1f9, 62fc05525542484e8baa5885fab02627; no voters:
I20260923 18:00:43.285346 23891 raft_consensus.cc:2454] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62fc05525542484e8baa5885fab02627 in term 1.
I20260923 18:00:43.285666 23994 raft_consensus.cc:2781] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:43.285851 23994 raft_consensus.cc:686] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 1 LEADER]: Becoming Leader. State: Replica: 62fc05525542484e8baa5885fab02627, State: Running, Role: LEADER
I20260923 18:00:43.286111 23994 consensus_queue.cc:237] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.288641 23451 catalog_manager.cc:5495] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62fc05525542484e8baa5885fab02627 (127.15.139.129). New cstate: current_term: 1 leader_uuid: "62fc05525542484e8baa5885fab02627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } health_report { overall_health: UNKNOWN } } }
Sep 23 18:00:43 dist-test-slave-vr57 krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186442, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.190@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:43.330725 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.321435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:37677 (local address 127.0.0.1:50454)
0923 18:00:43.321845 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.321858 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.322004 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.322283 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.322286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.322728 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:43.322829 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.322836 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.323686 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.323690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.324269 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.324277 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.324741 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.324745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.324846 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.325551 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:43.325564 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:43.327270 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:43.329093 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:43.329105 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:43.329112 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:43.329339 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:43.329634 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:43.329639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:43.329641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:43.329720 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:43.330107 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:43.330115 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:43.330222 (+ 107us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.330331 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.330485 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":113,"threads_started":1}
I20260923 18:00:43.330725 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.321518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50454 (local address 127.15.139.190:37677)
0923 18:00:43.321686 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.321691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.321940 (+ 249us) server_negotiation.cc:407] Connection header received
0923 18:00:43.322105 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.322109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.322161 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.322262 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:43.322958 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.323561 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.324391 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.324625 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.324739 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.327449 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:43.327452 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:43.327454 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:43.327474 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:43.328989 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:43.329476 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:43.329479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:43.329480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:43.329548 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:43.329805 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:43.329810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:43.329812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:43.329994 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:43.330096 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.330389 (+ 293us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.330536 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":72}
I20260923 18:00:43.334887 23451 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:50454:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 18:00:43.335791 23451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:43.341223 23591 tablet_service.cc:1467] Processing CreateTablet for tablet d7da3bdbc8ae444b88d6624137d8609a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3218583dbe1b4ecab948b2fb69348623]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 18:00:43.341323 23871 tablet_service.cc:1467] Processing CreateTablet for tablet d7da3bdbc8ae444b88d6624137d8609a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3218583dbe1b4ecab948b2fb69348623]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 18:00:43.341662 23591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7da3bdbc8ae444b88d6624137d8609a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.341759 23871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7da3bdbc8ae444b88d6624137d8609a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.344167 23965 tablet_bootstrap.cc:492] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d: Bootstrap starting.
I20260923 18:00:43.345074 23966 tablet_bootstrap.cc:492] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: Bootstrap starting.
I20260923 18:00:43.345283 23965 tablet_bootstrap.cc:654] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.346030 23966 tablet_bootstrap.cc:654] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.346614 23965 tablet_bootstrap.cc:492] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d: No bootstrap required, opened a new log
I20260923 18:00:43.346725 23965 ts_tablet_manager.cc:1397] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:43.347020 23731 tablet_service.cc:1467] Processing CreateTablet for tablet d7da3bdbc8ae444b88d6624137d8609a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3218583dbe1b4ecab948b2fb69348623]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 18:00:43.347208 23965 raft_consensus.cc:348] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.347301 23965 raft_consensus.cc:374] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.347324 23965 raft_consensus.cc:729] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fe794d6c76416abe4b47bf7ef90b3d, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.347374 23731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7da3bdbc8ae444b88d6624137d8609a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.347460 23965 consensus_queue.cc:260] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.347833 23965 ts_tablet_manager.cc:1428] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:43.347774 23985 tablet_replica.cc:439] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } } }
I20260923 18:00:43.347909 23985 tablet_replica.cc:442] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:43.350596 23967 tablet_bootstrap.cc:492] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9: Bootstrap starting.
I20260923 18:00:43.351446 23966 tablet_bootstrap.cc:492] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: No bootstrap required, opened a new log
I20260923 18:00:43.351563 23966 ts_tablet_manager.cc:1397] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260923 18:00:43.351719 23967 tablet_bootstrap.cc:654] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.352046 23966 raft_consensus.cc:348] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.352167 23966 raft_consensus.cc:374] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.352202 23966 raft_consensus.cc:729] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fc05525542484e8baa5885fab02627, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.352339 23966 consensus_queue.cc:260] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.352782 23994 tablet_replica.cc:439] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } } }
I20260923 18:00:43.352921 23994 tablet_replica.cc:442] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:43.353498 23966 ts_tablet_manager.cc:1428] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:43.354642 23967 tablet_bootstrap.cc:492] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9: No bootstrap required, opened a new log
I20260923 18:00:43.354739 23967 ts_tablet_manager.cc:1397] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260923 18:00:43.355178 23967 raft_consensus.cc:348] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.355301 23967 raft_consensus.cc:374] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.355335 23967 raft_consensus.cc:729] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3617e62d4a45dc9a021fe4a1eff1f9, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.355521 23967 consensus_queue.cc:260] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.355762 23972 tablet_replica.cc:439] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } } }
I20260923 18:00:43.355886 23972 tablet_replica.cc:442] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:43.356509 23967 ts_tablet_manager.cc:1428] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:43.415313 23994 raft_consensus.cc:482] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:43.415493 23994 raft_consensus.cc:504] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.416086 23994 leader_election.cc:288] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837), 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131:32975)
I20260923 18:00:43.416528 23751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7da3bdbc8ae444b88d6624137d8609a" candidate_uuid: "62fc05525542484e8baa5885fab02627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" is_pre_election: true
I20260923 18:00:43.416538 23891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7da3bdbc8ae444b88d6624137d8609a" candidate_uuid: "62fc05525542484e8baa5885fab02627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" is_pre_election: true
I20260923 18:00:43.416693 23891 raft_consensus.cc:2454] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62fc05525542484e8baa5885fab02627 in term 0.
I20260923 18:00:43.416707 23751 raft_consensus.cc:2454] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62fc05525542484e8baa5885fab02627 in term 0.
I20260923 18:00:43.417038 23542 leader_election.cc:302] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62fc05525542484e8baa5885fab02627, 93fe794d6c76416abe4b47bf7ef90b3d; no voters:
I20260923 18:00:43.417286 23994 raft_consensus.cc:2781] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:43.417351 23994 raft_consensus.cc:482] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:43.417407 23994 raft_consensus.cc:3037] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.418403 23994 raft_consensus.cc:504] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.418771 23994 leader_election.cc:288] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [CANDIDATE]: Term 1 election: Requested vote from peers 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837), 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131:32975)
I20260923 18:00:43.419154 23751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7da3bdbc8ae444b88d6624137d8609a" candidate_uuid: "62fc05525542484e8baa5885fab02627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9"
I20260923 18:00:43.419143 23891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7da3bdbc8ae444b88d6624137d8609a" candidate_uuid: "62fc05525542484e8baa5885fab02627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fe794d6c76416abe4b47bf7ef90b3d"
I20260923 18:00:43.419281 23891 raft_consensus.cc:3037] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.419286 23751 raft_consensus.cc:3037] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.420248 23751 raft_consensus.cc:2454] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62fc05525542484e8baa5885fab02627 in term 1.
I20260923 18:00:43.420524 23891 raft_consensus.cc:2454] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62fc05525542484e8baa5885fab02627 in term 1.
I20260923 18:00:43.420559 23545 leader_election.cc:302] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3617e62d4a45dc9a021fe4a1eff1f9, 62fc05525542484e8baa5885fab02627; no voters:
I20260923 18:00:43.420713 23994 raft_consensus.cc:2781] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:43.420799 23994 raft_consensus.cc:686] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [term 1 LEADER]: Becoming Leader. State: Replica: 62fc05525542484e8baa5885fab02627, State: Running, Role: LEADER
I20260923 18:00:43.420931 23994 consensus_queue.cc:237] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.421658 23973 tablet_replica.cc:439] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: TxnStatusTablet state changed. Reason: New leader 62fc05525542484e8baa5885fab02627. Latest consensus state: current_term: 1 leader_uuid: "62fc05525542484e8baa5885fab02627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } } }
I20260923 18:00:43.421772 23973 tablet_replica.cc:442] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: This TxnStatusTablet replica's current role is: LEADER
I20260923 18:00:43.422080 24008 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 18:00:43.422290 24008 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 18:00:43.423105 23453 catalog_manager.cc:5495] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62fc05525542484e8baa5885fab02627 (127.15.139.129). New cstate: current_term: 1 leader_uuid: "62fc05525542484e8baa5885fab02627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } health_report { overall_health: UNKNOWN } } }
I20260923 18:00:43.438265 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.432465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35227 (local address 127.0.0.1:37766)
0923 18:00:43.432842 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.432859 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.432980 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.433448 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.433451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.433460 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 18:00:43.433567 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.433574 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.434744 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.434748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.435942 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.435958 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.436726 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.436730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.436828 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.436883 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 18:00:43.437917 (+ 1034us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 18:00:43.437933 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.438013 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.438104 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276}
I20260923 18:00:43.438824 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.432580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37766 (local address 127.15.139.129:35227)
0923 18:00:43.432708 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.432713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.433095 (+ 382us) server_negotiation.cc:407] Connection header received
0923 18:00:43.433168 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.433171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.433233 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.433326 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:43.433704 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.434587 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.436139 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.436519 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.436631 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.437077 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:43.437747 (+ 670us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:43.437869 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.438623 (+ 754us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.438687 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 18:00:43.441962 23751 raft_consensus.cc:1261] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Refusing update from remote peer 62fc05525542484e8baa5885fab02627: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:43.442024 23891 raft_consensus.cc:1261] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Refusing update from remote peer 62fc05525542484e8baa5885fab02627: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:43.442514 23973 consensus_queue.cc:1035] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:43.442679 23994 consensus_queue.cc:1035] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:43.445418 23972 tablet_replica.cc:439] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9: TxnStatusTablet state changed. Reason: New leader 62fc05525542484e8baa5885fab02627. Latest consensus state: current_term: 1 leader_uuid: "62fc05525542484e8baa5885fab02627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } } }
I20260923 18:00:43.445554 23972 tablet_replica.cc:442] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:43.445894 23994 tablet_replica.cc:439] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62fc05525542484e8baa5885fab02627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } } }
I20260923 18:00:43.446017 23994 tablet_replica.cc:442] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: This TxnStatusTablet replica's current role is: LEADER
I20260923 18:00:43.448608 23985 tablet_replica.cc:439] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d: TxnStatusTablet state changed. Reason: New leader 62fc05525542484e8baa5885fab02627. Latest consensus state: current_term: 1 leader_uuid: "62fc05525542484e8baa5885fab02627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } } }
I20260923 18:00:43.448736 23985 tablet_replica.cc:442] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:43.449240 23972 tablet_replica.cc:439] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62fc05525542484e8baa5885fab02627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } } }
I20260923 18:00:43.449349 23972 tablet_replica.cc:442] T d7da3bdbc8ae444b88d6624137d8609a P 0e3617e62d4a45dc9a021fe4a1eff1f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:43.455336 23973 tablet_replica.cc:439] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62fc05525542484e8baa5885fab02627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } } }
I20260923 18:00:43.455477 23973 tablet_replica.cc:442] T d7da3bdbc8ae444b88d6624137d8609a P 62fc05525542484e8baa5885fab02627: This TxnStatusTablet replica's current role is: LEADER
I20260923 18:00:43.456759 23985 tablet_replica.cc:439] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62fc05525542484e8baa5885fab02627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } } }
I20260923 18:00:43.456879 23985 tablet_replica.cc:442] T d7da3bdbc8ae444b88d6624137d8609a P 93fe794d6c76416abe4b47bf7ef90b3d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 18:00:43.460881 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.454064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37776 (local address 127.15.139.129:35227)
0923 18:00:43.454177 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.454182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.457370 (+ 3188us) server_negotiation.cc:407] Connection header received
0923 18:00:43.457423 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.457426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.457497 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.457580 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:43.457845 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.458558 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.459688 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.460027 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.460152 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.460423 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:43.460510 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:43.460592 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.460694 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.460745 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 18:00:43.472153 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.468422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:42525 (local address 127.15.139.129:35227)
0923 18:00:43.468566 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.468571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.468839 (+ 268us) server_negotiation.cc:407] Connection header received
0923 18:00:43.469005 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.469008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.469052 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.469129 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:43.469493 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.470023 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.470954 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.471195 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.471306 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.471667 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:43.471742 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:43.471822 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.471909 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.471976 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":51}
I20260923 18:00:43.472153 24015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.468389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35227 (local address 127.15.139.129:42525)
0923 18:00:43.468760 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.468773 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.468890 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.469148 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.469151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.469158 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 18:00:43.469379 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.469391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.470126 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.470133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.470835 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.470843 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.471303 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.471307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.471432 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.471480 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 18:00:43.471823 (+ 343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 18:00:43.471830 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.471901 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.472012 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":91,"thread_start_us":119,"threads_started":1}
I20260923 18:00:43.480067 23751 raft_consensus.cc:1261] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Refusing update from remote peer 62fc05525542484e8baa5885fab02627: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:43.480517 24016 consensus_queue.cc:1035] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:43.480563 23891 raft_consensus.cc:1261] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Refusing update from remote peer 62fc05525542484e8baa5885fab02627: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:43.480997 24016 consensus_queue.cc:1035] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:43.525386 23451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:43.525578 23451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:43.527667 23451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4: Sending DeleteTablet for 3 replicas of tablet 267822f27de64864b4d45634cd6a9691
I20260923 18:00:43.528225 23590 tablet_service.cc:1514] Processing DeleteTablet for tablet 267822f27de64864b4d45634cd6a9691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:37758
I20260923 18:00:43.528445 23988 tablet_replica.cc:330] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627: stopping tablet replica
I20260923 18:00:43.528584 23988 raft_consensus.cc:2229] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:43.528594 23731 tablet_service.cc:1514] Processing DeleteTablet for tablet 267822f27de64864b4d45634cd6a9691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:48618
I20260923 18:00:43.528577 23871 tablet_service.cc:1514] Processing DeleteTablet for tablet 267822f27de64864b4d45634cd6a9691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:49092
I20260923 18:00:43.528769 23990 tablet_replica.cc:330] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9: stopping tablet replica
I20260923 18:00:43.528785 23989 tablet_replica.cc:330] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d: stopping tablet replica
I20260923 18:00:43.528882 23990 raft_consensus.cc:2229] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:43.528906 23989 raft_consensus.cc:2229] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:43.528923 23988 raft_consensus.cc:2258] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:43.529013 23990 raft_consensus.cc:2258] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:43.529057 23989 raft_consensus.cc:2258] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:43.529963 23990 ts_tablet_manager.cc:1905] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:43.529963 23989 ts_tablet_manager.cc:1905] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:43.532181 23988 ts_tablet_manager.cc:1905] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:43.533075 23989 ts_tablet_manager.cc:1918] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:43.533150 23989 log.cc:1196] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/267822f27de64864b4d45634cd6a9691
I20260923 18:00:43.533427 23989 ts_tablet_manager.cc:1939] T 267822f27de64864b4d45634cd6a9691 P 93fe794d6c76416abe4b47bf7ef90b3d: Deleting consensus metadata
I20260923 18:00:43.534192 23438 catalog_manager.cc:4841] TS 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131:32975): tablet 267822f27de64864b4d45634cd6a9691 (table test-table [id=0b6c66f62d534e3a87844e2571c2844c]) successfully deleted
I20260923 18:00:43.535174 23988 ts_tablet_manager.cc:1918] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:43.535270 23988 log.cc:1196] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/267822f27de64864b4d45634cd6a9691
I20260923 18:00:43.535614 23988 ts_tablet_manager.cc:1939] T 267822f27de64864b4d45634cd6a9691 P 62fc05525542484e8baa5885fab02627: Deleting consensus metadata
I20260923 18:00:43.536258 23990 ts_tablet_manager.cc:1918] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:43.536293 23439 catalog_manager.cc:4841] TS 62fc05525542484e8baa5885fab02627 (127.15.139.129:35227): tablet 267822f27de64864b4d45634cd6a9691 (table test-table [id=0b6c66f62d534e3a87844e2571c2844c]) successfully deleted
I20260923 18:00:43.536340 23990 log.cc:1196] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/267822f27de64864b4d45634cd6a9691
I20260923 18:00:43.536618 23990 ts_tablet_manager.cc:1939] T 267822f27de64864b4d45634cd6a9691 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Deleting consensus metadata
I20260923 18:00:43.537290 23441 catalog_manager.cc:4841] TS 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837): tablet 267822f27de64864b4d45634cd6a9691 (table test-table [id=0b6c66f62d534e3a87844e2571c2844c]) successfully deleted
Sep 23 18:00:43 dist-test-slave-vr57 krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 18:00:43 dist-test-slave-vr57 krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:43 dist-test-slave-vr57 krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 18:00:43.561638 23503 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:43.554617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50460 (local address 127.15.139.190:37677)
0923 18:00:43.554768 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.554773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.554969 (+ 196us) server_negotiation.cc:407] Connection header received
0923 18:00:43.555148 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.555150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.555193 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.555277 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:43.556122 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.556645 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.557505 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.557819 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.557925 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.561471 (+ 3546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50460: BlockingRecv error: recv got EOF from 127.0.0.1:50460 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 23 18:00:43 dist-test-slave-vr57 krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM
I20260923 18:00:43.571615 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.563147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50474 (local address 127.15.139.190:37677)
0923 18:00:43.563259 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.563263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.563425 (+ 162us) server_negotiation.cc:407] Connection header received
0923 18:00:43.563606 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.563609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.563656 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.563738 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:43.564304 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.564808 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.565573 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.565795 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.565906 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.568603 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:43.568607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:43.568610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:43.568639 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:43.570000 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:43.570430 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:43.570434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:43.570436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:43.570511 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:43.570772 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:43.570776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:43.570778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:43.570954 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:43.571099 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.571319 (+ 220us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.571431 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 18:00:43.574100 23451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:00:43.574548 23451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:43.579457 23590 tablet_service.cc:1467] Processing CreateTablet for tablet f20562ad8ad3486f8624ab5a0f29ee80 (DEFAULT_TABLE table=test-table [id=454898f0c8884b058f108633d4863548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:43.579723 23871 tablet_service.cc:1467] Processing CreateTablet for tablet f20562ad8ad3486f8624ab5a0f29ee80 (DEFAULT_TABLE table=test-table [id=454898f0c8884b058f108633d4863548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:43.579864 23590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f20562ad8ad3486f8624ab5a0f29ee80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.580029 23871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f20562ad8ad3486f8624ab5a0f29ee80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.582063 23965 tablet_bootstrap.cc:492] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d: Bootstrap starting.
I20260923 18:00:43.582580 23966 tablet_bootstrap.cc:492] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627: Bootstrap starting.
I20260923 18:00:43.582890 23731 tablet_service.cc:1467] Processing CreateTablet for tablet f20562ad8ad3486f8624ab5a0f29ee80 (DEFAULT_TABLE table=test-table [id=454898f0c8884b058f108633d4863548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:43.582983 23965 tablet_bootstrap.cc:654] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.583180 23731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f20562ad8ad3486f8624ab5a0f29ee80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.583585 23966 tablet_bootstrap.cc:654] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.584703 23965 tablet_bootstrap.cc:492] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d: No bootstrap required, opened a new log
I20260923 18:00:43.584703 23966 tablet_bootstrap.cc:492] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627: No bootstrap required, opened a new log
I20260923 18:00:43.584786 23966 ts_tablet_manager.cc:1397] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 18:00:43.584786 23965 ts_tablet_manager.cc:1397] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 18:00:43.585197 23965 raft_consensus.cc:348] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.585196 23966 raft_consensus.cc:348] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.585307 23966 raft_consensus.cc:374] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.585307 23965 raft_consensus.cc:374] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.585341 23966 raft_consensus.cc:729] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fc05525542484e8baa5885fab02627, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.585341 23965 raft_consensus.cc:729] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fe794d6c76416abe4b47bf7ef90b3d, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.585469 23965 consensus_queue.cc:260] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.585469 23966 consensus_queue.cc:260] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.585695 23967 tablet_bootstrap.cc:492] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Bootstrap starting.
I20260923 18:00:43.585729 23965 ts_tablet_manager.cc:1428] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:43.585739 23966 ts_tablet_manager.cc:1428] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260923 18:00:43.586606 23967 tablet_bootstrap.cc:654] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.588088 23967 tablet_bootstrap.cc:492] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9: No bootstrap required, opened a new log
I20260923 18:00:43.588173 23967 ts_tablet_manager.cc:1397] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:43.588644 23967 raft_consensus.cc:348] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.588757 23967 raft_consensus.cc:374] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.588792 23967 raft_consensus.cc:729] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3617e62d4a45dc9a021fe4a1eff1f9, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.588912 23967 consensus_queue.cc:260] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.589149 23967 ts_tablet_manager.cc:1428] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:43.692070 23985 raft_consensus.cc:482] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:43.692234 23985 raft_consensus.cc:504] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.692764 23985 leader_election.cc:288] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62fc05525542484e8baa5885fab02627 (127.15.139.129:35227), 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837)
I20260923 18:00:43.693286 23611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f20562ad8ad3486f8624ab5a0f29ee80" candidate_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc05525542484e8baa5885fab02627" is_pre_election: true
I20260923 18:00:43.693288 23751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f20562ad8ad3486f8624ab5a0f29ee80" candidate_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" is_pre_election: true
I20260923 18:00:43.693490 23751 raft_consensus.cc:2454] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93fe794d6c76416abe4b47bf7ef90b3d in term 0.
I20260923 18:00:43.693491 23611 raft_consensus.cc:2454] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93fe794d6c76416abe4b47bf7ef90b3d in term 0.
I20260923 18:00:43.693807 23823 leader_election.cc:302] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62fc05525542484e8baa5885fab02627, 93fe794d6c76416abe4b47bf7ef90b3d; no voters:
I20260923 18:00:43.694026 23985 raft_consensus.cc:2781] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:43.694106 23985 raft_consensus.cc:482] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:43.694170 23985 raft_consensus.cc:3037] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.695385 23985 raft_consensus.cc:504] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.695812 23985 leader_election.cc:288] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [CANDIDATE]: Term 1 election: Requested vote from peers 62fc05525542484e8baa5885fab02627 (127.15.139.129:35227), 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837)
I20260923 18:00:43.696175 23611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f20562ad8ad3486f8624ab5a0f29ee80" candidate_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc05525542484e8baa5885fab02627"
I20260923 18:00:43.696194 23751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f20562ad8ad3486f8624ab5a0f29ee80" candidate_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9"
I20260923 18:00:43.696313 23751 raft_consensus.cc:3037] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.696317 23611 raft_consensus.cc:3037] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.697281 23611 raft_consensus.cc:2454] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93fe794d6c76416abe4b47bf7ef90b3d in term 1.
I20260923 18:00:43.697418 23751 raft_consensus.cc:2454] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93fe794d6c76416abe4b47bf7ef90b3d in term 1.
I20260923 18:00:43.697659 23823 leader_election.cc:302] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62fc05525542484e8baa5885fab02627, 93fe794d6c76416abe4b47bf7ef90b3d; no voters:
I20260923 18:00:43.697858 23985 raft_consensus.cc:2781] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:43.697973 23985 raft_consensus.cc:686] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 LEADER]: Becoming Leader. State: Replica: 93fe794d6c76416abe4b47bf7ef90b3d, State: Running, Role: LEADER
I20260923 18:00:43.698134 23985 consensus_queue.cc:237] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.700068 23451 catalog_manager.cc:5495] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131). New cstate: current_term: 1 leader_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } health_report { overall_health: HEALTHY } } }
I20260923 18:00:43.716655 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.713316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49116 (local address 127.15.139.131:32975)
0923 18:00:43.713461 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.713466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.713482 (+ 16us) server_negotiation.cc:407] Connection header received
0923 18:00:43.713533 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.713537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.713585 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.713674 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:43.713899 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.714538 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.715618 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.715853 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.715962 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.716223 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:43.716301 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:43.716375 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.716464 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.716520 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 18:00:43.718673 23751 raft_consensus.cc:1261] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Refusing update from remote peer 93fe794d6c76416abe4b47bf7ef90b3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:43.718675 23611 raft_consensus.cc:1261] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Refusing update from remote peer 93fe794d6c76416abe4b47bf7ef90b3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:43.719154 23985 consensus_queue.cc:1035] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:43.719477 23985 consensus_queue.cc:1035] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:43.728415 23453 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:43.728564 23453 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:43.731092 23453 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4: Sending DeleteTablet for 3 replicas of tablet f20562ad8ad3486f8624ab5a0f29ee80
I20260923 18:00:43.731792 23731 tablet_service.cc:1514] Processing DeleteTablet for tablet f20562ad8ad3486f8624ab5a0f29ee80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:48618
I20260923 18:00:43.731812 23590 tablet_service.cc:1514] Processing DeleteTablet for tablet f20562ad8ad3486f8624ab5a0f29ee80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:37758
I20260923 18:00:43.731809 23730 tablet_service.cc:1514] Processing DeleteTablet for tablet f20562ad8ad3486f8624ab5a0f29ee80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:48618
I20260923 18:00:43.731992 23988 tablet_replica.cc:330] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627: stopping tablet replica
I20260923 18:00:43.731990 23990 tablet_replica.cc:330] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9: stopping tablet replica
I20260923 18:00:43.732101 23990 raft_consensus.cc:2229] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:43.732120 23988 raft_consensus.cc:2229] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:43.732234 23990 raft_consensus.cc:2258] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:43.732261 23988 raft_consensus.cc:2258] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:43.733029 23990 ts_tablet_manager.cc:1905] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:43.733683 23871 tablet_service.cc:1514] Processing DeleteTablet for tablet f20562ad8ad3486f8624ab5a0f29ee80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:49092
I20260923 18:00:43.733868 23989 tablet_replica.cc:330] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d: stopping tablet replica
I20260923 18:00:43.734004 23989 raft_consensus.cc:2229] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:43.734223 23989 raft_consensus.cc:2258] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:43.734896 23989 ts_tablet_manager.cc:1905] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:43.735556 23990 ts_tablet_manager.cc:1918] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:43.735642 23990 log.cc:1196] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/f20562ad8ad3486f8624ab5a0f29ee80
I20260923 18:00:43.735918 23990 ts_tablet_manager.cc:1939] T f20562ad8ad3486f8624ab5a0f29ee80 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Deleting consensus metadata
I20260923 18:00:43.736692 23441 catalog_manager.cc:4841] TS 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837): tablet f20562ad8ad3486f8624ab5a0f29ee80 (table test-table [id=454898f0c8884b058f108633d4863548]) successfully deleted
W20260923 18:00:43.737026 23441 catalog_manager.cc:4808] TS 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837): delete failed for tablet f20562ad8ad3486f8624ab5a0f29ee80 because the tablet was not found. No further retry: Not found: Tablet not found: f20562ad8ad3486f8624ab5a0f29ee80
I20260923 18:00:43.737255 23989 ts_tablet_manager.cc:1918] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:43.737332 23989 log.cc:1196] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/f20562ad8ad3486f8624ab5a0f29ee80
I20260923 18:00:43.737567 23989 ts_tablet_manager.cc:1939] T f20562ad8ad3486f8624ab5a0f29ee80 P 93fe794d6c76416abe4b47bf7ef90b3d: Deleting consensus metadata
I20260923 18:00:43.738193 23438 catalog_manager.cc:4841] TS 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131:32975): tablet f20562ad8ad3486f8624ab5a0f29ee80 (table test-table [id=454898f0c8884b058f108633d4863548]) successfully deleted
I20260923 18:00:43.739056 23453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:00:43.739490 23453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:00:43.740624 23988 ts_tablet_manager.cc:1905] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:43.742595 23988 ts_tablet_manager.cc:1918] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:00:43.742690 23988 log.cc:1196] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/f20562ad8ad3486f8624ab5a0f29ee80
I20260923 18:00:43.742928 23988 ts_tablet_manager.cc:1939] T f20562ad8ad3486f8624ab5a0f29ee80 P 62fc05525542484e8baa5885fab02627: Deleting consensus metadata
I20260923 18:00:43.743613 23439 catalog_manager.cc:4841] TS 62fc05525542484e8baa5885fab02627 (127.15.139.129:35227): tablet f20562ad8ad3486f8624ab5a0f29ee80 (table test-table [id=454898f0c8884b058f108633d4863548]) successfully deleted
I20260923 18:00:43.746770 23730 tablet_service.cc:1467] Processing CreateTablet for tablet f208c861a695457da27ea7f8b99adba0 (DEFAULT_TABLE table=test-table [id=e9793bb57f8947b9a83d10aa90e59d4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:43.746853 23590 tablet_service.cc:1467] Processing CreateTablet for tablet f208c861a695457da27ea7f8b99adba0 (DEFAULT_TABLE table=test-table [id=e9793bb57f8947b9a83d10aa90e59d4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:43.746975 23871 tablet_service.cc:1467] Processing CreateTablet for tablet f208c861a695457da27ea7f8b99adba0 (DEFAULT_TABLE table=test-table [id=e9793bb57f8947b9a83d10aa90e59d4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:00:43.747089 23730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f208c861a695457da27ea7f8b99adba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.747138 23590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f208c861a695457da27ea7f8b99adba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.747233 23871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f208c861a695457da27ea7f8b99adba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:43.749676 23965 tablet_bootstrap.cc:492] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d: Bootstrap starting.
I20260923 18:00:43.749693 23967 tablet_bootstrap.cc:492] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Bootstrap starting.
I20260923 18:00:43.749764 23966 tablet_bootstrap.cc:492] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627: Bootstrap starting.
I20260923 18:00:43.750547 23965 tablet_bootstrap.cc:654] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.750558 23967 tablet_bootstrap.cc:654] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.750547 23966 tablet_bootstrap.cc:654] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:43.752102 23965 tablet_bootstrap.cc:492] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d: No bootstrap required, opened a new log
I20260923 18:00:43.752142 23967 tablet_bootstrap.cc:492] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9: No bootstrap required, opened a new log
I20260923 18:00:43.752194 23965 ts_tablet_manager.cc:1397] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:43.752213 23967 ts_tablet_manager.cc:1397] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:43.752612 23967 raft_consensus.cc:348] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.752712 23967 raft_consensus.cc:374] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.752651 23965 raft_consensus.cc:348] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.752740 23967 raft_consensus.cc:729] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3617e62d4a45dc9a021fe4a1eff1f9, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.752756 23965 raft_consensus.cc:374] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.752792 23965 raft_consensus.cc:729] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fe794d6c76416abe4b47bf7ef90b3d, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.752848 23967 consensus_queue.cc:260] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.752883 23966 tablet_bootstrap.cc:492] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627: No bootstrap required, opened a new log
I20260923 18:00:43.752955 23966 ts_tablet_manager.cc:1397] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260923 18:00:43.752914 23965 consensus_queue.cc:260] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.753070 23967 ts_tablet_manager.cc:1428] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:43.753178 23965 ts_tablet_manager.cc:1428] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:43.753353 23966 raft_consensus.cc:348] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.753463 23966 raft_consensus.cc:374] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:43.753494 23966 raft_consensus.cc:729] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fc05525542484e8baa5885fab02627, State: Initialized, Role: FOLLOWER
I20260923 18:00:43.753618 23966 consensus_queue.cc:260] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.753886 23966 ts_tablet_manager.cc:1428] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:00:43.827782 23972 raft_consensus.cc:482] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:00:43.827926 23972 raft_consensus.cc:504] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.829221 23972 leader_election.cc:288] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62fc05525542484e8baa5885fab02627 (127.15.139.129:35227), 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131:32975)
I20260923 18:00:43.833323 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.829481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:32975 (local address 127.15.139.130:41065)
0923 18:00:43.829817 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.829834 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.829940 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.830228 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.830231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.830244 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:43.830371 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.830378 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.831205 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.831212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.832229 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.832239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.832878 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.832882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.832989 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.833016 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.833078 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.833136 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":65,"thread_start_us":90,"threads_started":1}
I20260923 18:00:43.833334 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.829768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:41065 (local address 127.15.139.131:32975)
0923 18:00:43.829893 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.829899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.829916 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:00:43.830078 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.830081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.830133 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.830213 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:43.830538 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.831085 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.832347 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.832753 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.832872 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.832933 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.833112 (+ 179us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.833205 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 18:00:43.833422 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.829620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:34291 (local address 127.15.139.129:35227)
0923 18:00:43.829755 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.829760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.830038 (+ 278us) server_negotiation.cc:407] Connection header received
0923 18:00:43.830216 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.830219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.830268 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.830337 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:00:43.830658 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.831151 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.832379 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.832776 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.832915 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.832966 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.833212 (+ 246us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.833291 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:00:43.833382 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.829496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35227 (local address 127.15.139.130:34291)
0923 18:00:43.829912 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.829923 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.830106 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.830357 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.830360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.830379 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:00:43.830539 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.830546 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.831266 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.831269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.832236 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.832246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.832944 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.832951 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.833037 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.833053 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.833176 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.833240 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":51,"thread_start_us":81,"threads_started":1}
I20260923 18:00:43.833832 23891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f208c861a695457da27ea7f8b99adba0" candidate_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" is_pre_election: true
I20260923 18:00:43.834026 23891 raft_consensus.cc:2454] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e3617e62d4a45dc9a021fe4a1eff1f9 in term 0.
I20260923 18:00:43.834003 23611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f208c861a695457da27ea7f8b99adba0" candidate_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc05525542484e8baa5885fab02627" is_pre_election: true
I20260923 18:00:43.834137 23611 raft_consensus.cc:2454] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e3617e62d4a45dc9a021fe4a1eff1f9 in term 0.
I20260923 18:00:43.834412 23682 leader_election.cc:302] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3617e62d4a45dc9a021fe4a1eff1f9, 93fe794d6c76416abe4b47bf7ef90b3d; no voters:
I20260923 18:00:43.834693 23972 raft_consensus.cc:2781] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:00:43.834765 23972 raft_consensus.cc:482] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:00:43.834829 23972 raft_consensus.cc:3037] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.835938 23972 raft_consensus.cc:504] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.836293 23972 leader_election.cc:288] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [CANDIDATE]: Term 1 election: Requested vote from peers 62fc05525542484e8baa5885fab02627 (127.15.139.129:35227), 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131:32975)
I20260923 18:00:43.836639 23611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f208c861a695457da27ea7f8b99adba0" candidate_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc05525542484e8baa5885fab02627"
I20260923 18:00:43.836745 23611 raft_consensus.cc:3037] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.836685 23891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f208c861a695457da27ea7f8b99adba0" candidate_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fe794d6c76416abe4b47bf7ef90b3d"
I20260923 18:00:43.836850 23891 raft_consensus.cc:3037] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:43.837697 23611 raft_consensus.cc:2454] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3617e62d4a45dc9a021fe4a1eff1f9 in term 1.
I20260923 18:00:43.837808 23891 raft_consensus.cc:2454] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3617e62d4a45dc9a021fe4a1eff1f9 in term 1.
I20260923 18:00:43.838012 23683 leader_election.cc:302] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3617e62d4a45dc9a021fe4a1eff1f9, 62fc05525542484e8baa5885fab02627; no voters:
I20260923 18:00:43.838179 23972 raft_consensus.cc:2781] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:43.838404 23972 raft_consensus.cc:686] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 LEADER]: Becoming Leader. State: Replica: 0e3617e62d4a45dc9a021fe4a1eff1f9, State: Running, Role: LEADER
I20260923 18:00:43.838680 23972 consensus_queue.cc:237] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } } peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } }
I20260923 18:00:43.841038 23452 catalog_manager.cc:5495] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130). New cstate: current_term: 1 leader_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3617e62d4a45dc9a021fe4a1eff1f9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 37837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 } health_report { overall_health: UNKNOWN } } }
I20260923 18:00:43.863610 24047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.860005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48626 (local address 127.15.139.130:37837)
0923 18:00:43.860230 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.860246 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.860258 (+ 12us) server_negotiation.cc:407] Connection header received
0923 18:00:43.860364 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.860367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.860421 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.860528 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:43.860720 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.861353 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.862176 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.862435 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.862516 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.862742 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:43.863225 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:43.863314 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.863372 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.863420 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":99,"threads_started":1}
I20260923 18:00:43.874560 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.870806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35227 (local address 127.15.139.130:38581)
0923 18:00:43.871126 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.871139 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.871237 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.871439 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.871441 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.871448 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 18:00:43.871991 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.871998 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.872700 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.872702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.873336 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.873343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.873811 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.873815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.873902 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.873946 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 18:00:43.874246 (+ 300us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 18:00:43.874254 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.874333 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.874414 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":110,"threads_started":1}
I20260923 18:00:43.874552 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.870965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:38581 (local address 127.15.139.129:35227)
0923 18:00:43.871111 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.871114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.871206 (+ 92us) server_negotiation.cc:407] Connection header received
0923 18:00:43.871310 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.871312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.871355 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.871424 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:43.872097 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.872604 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.873469 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.873698 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.873808 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.874093 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:43.874165 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:43.874239 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.874345 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.874405 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 18:00:43.881477 23891 raft_consensus.cc:1261] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Refusing update from remote peer 0e3617e62d4a45dc9a021fe4a1eff1f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:43.881441 23611 raft_consensus.cc:1261] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Refusing update from remote peer 0e3617e62d4a45dc9a021fe4a1eff1f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:00:43.882023 24045 consensus_queue.cc:1035] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93fe794d6c76416abe4b47bf7ef90b3d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 32975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:43.882225 23972 consensus_queue.cc:1035] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62fc05525542484e8baa5885fab02627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:00:43.899657 24047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.896349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:56031 (local address 127.15.139.130:37837)
0923 18:00:43.896454 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:43.896462 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:43.896499 (+ 37us) server_negotiation.cc:407] Connection header received
0923 18:00:43.896673 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:43.896676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:43.896730 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:43.896804 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:00:43.897064 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.897621 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.898409 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.898741 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.898874 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.899093 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:00:43.899188 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:00:43.899286 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:43.899424 (+ 138us) server_negotiation.cc:299] Negotiation successful
0923 18:00:43.899482 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 18:00:43.899657 24015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.896290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:37837 (local address 127.15.139.129:56031)
0923 18:00:43.896430 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.896448 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:43.896557 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:43.896850 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:43.896855 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:43.896865 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 18:00:43.896975 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.896981 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.897713 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.897716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.898292 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:43.898299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:43.898877 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:43.898880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:43.898944 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:43.898974 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 18:00:43.899328 (+ 354us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 18:00:43.899337 (+ 9us) client_negotiation.cc:769] Sending connection context
0923 18:00:43.899402 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 18:00:43.899521 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 18:00:43.920302 23452 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:43.920496 23452 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:00:43.922935 23452 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 732231d5680042a88a4f160cddcabec4: Sending DeleteTablet for 3 replicas of tablet f208c861a695457da27ea7f8b99adba0
I20260923 18:00:43.923620 23730 tablet_service.cc:1514] Processing DeleteTablet for tablet f208c861a695457da27ea7f8b99adba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:48618
I20260923 18:00:43.923825 23990 tablet_replica.cc:330] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9: stopping tablet replica
I20260923 18:00:43.923795 23871 tablet_service.cc:1514] Processing DeleteTablet for tablet f208c861a695457da27ea7f8b99adba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:49092
I20260923 18:00:43.923866 23590 tablet_service.cc:1514] Processing DeleteTablet for tablet f208c861a695457da27ea7f8b99adba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:00:43 UTC) from {username='oryx', principal='oryx/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:37758
I20260923 18:00:43.923947 23990 raft_consensus.cc:2229] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:00:43.923998 23989 tablet_replica.cc:330] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d: stopping tablet replica
I20260923 18:00:43.924113 23989 raft_consensus.cc:2229] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:43.924212 23989 raft_consensus.cc:2258] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:43.924283 23990 raft_consensus.cc:2258] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:43.924410 23988 tablet_replica.cc:330] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627: stopping tablet replica
I20260923 18:00:43.924528 23988 raft_consensus.cc:2229] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:00:43.924674 23988 raft_consensus.cc:2258] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:00:43.925244 23990 ts_tablet_manager.cc:1905] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:43.925413 23988 ts_tablet_manager.cc:1905] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:43.925407 23989 ts_tablet_manager.cc:1905] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:00:43.928066 23988 ts_tablet_manager.cc:1918] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:43.928067 23989 ts_tablet_manager.cc:1918] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:43.928179 23988 log.cc:1196] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/f208c861a695457da27ea7f8b99adba0
I20260923 18:00:43.928179 23989 log.cc:1196] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/f208c861a695457da27ea7f8b99adba0
I20260923 18:00:43.928471 23988 ts_tablet_manager.cc:1939] T f208c861a695457da27ea7f8b99adba0 P 62fc05525542484e8baa5885fab02627: Deleting consensus metadata
I20260923 18:00:43.928471 23989 ts_tablet_manager.cc:1939] T f208c861a695457da27ea7f8b99adba0 P 93fe794d6c76416abe4b47bf7ef90b3d: Deleting consensus metadata
I20260923 18:00:43.928705 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 23526
I20260923 18:00:43.931408 23438 catalog_manager.cc:4841] TS 93fe794d6c76416abe4b47bf7ef90b3d (127.15.139.131:32975): tablet f208c861a695457da27ea7f8b99adba0 (table test-table [id=e9793bb57f8947b9a83d10aa90e59d4c]) successfully deleted
I20260923 18:00:43.936403 23990 ts_tablet_manager.cc:1918] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 18:00:43.936509 23990 log.cc:1196] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/f208c861a695457da27ea7f8b99adba0
I20260923 18:00:43.936784 23990 ts_tablet_manager.cc:1939] T f208c861a695457da27ea7f8b99adba0 P 0e3617e62d4a45dc9a021fe4a1eff1f9: Deleting consensus metadata
I20260923 18:00:43.937558 23441 catalog_manager.cc:4841] TS 0e3617e62d4a45dc9a021fe4a1eff1f9 (127.15.139.130:37837): tablet f208c861a695457da27ea7f8b99adba0 (table test-table [id=e9793bb57f8947b9a83d10aa90e59d4c]) successfully deleted
W20260923 18:00:43.938444 23439 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:35227 (error 108)
I20260923 18:00:43.939000 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 23666
I20260923 18:00:43.940315 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:43.939893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35227 (local address 127.0.0.1:37784)
0923 18:00:43.940101 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:43.940187 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:35227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
W20260923 18:00:43.940534 23439 catalog_manager.cc:4584] TS 62fc05525542484e8baa5885fab02627 (127.15.139.129:35227): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f208c861a695457da27ea7f8b99adba0: Network error: Client connection negotiation failed: client connection to 127.15.139.129:35227: connect: Connection refused (error 111)
I20260923 18:00:43.946075 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 23806
I20260923 18:00:43.952744 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 23422
2026-09-23T18:00:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3185 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:44 dist-test-slave-vr57 krb5kdc[24059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:44 dist-test-slave-vr57 krb5kdc[24059](info): set up 2 sockets
Sep 23 18:00:44 dist-test-slave-vr57 krb5kdc[24059](info): commencing operation
krb5kdc: starting...
W20260923 18:00:45.036804 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:45 dist-test-slave-vr57 krb5kdc[24059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:00:45.161873 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:44483
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:39667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:44483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:45.274174 24072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:45.274518 24072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:45.274605 24072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:45.278798 24072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:45.278889 24072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:45.278934 24072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:45.278987 24072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:45.279027 24072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:45.284636 24072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:39667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:44483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:44483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:45.285862 24072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:45.286897 24072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:45.295485 24078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:45.295537 24072 server_base.cc:1034] running on GCE node
W20260923 18:00:45.295485 24077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:45.295545 24080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:45.296242 24072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:45.297384 24072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:45.298584 24072 hybrid_clock.cc:648] HybridClock initialized: now 1790186445298545 us; error 42 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-23T18:00:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1355 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:00:45 dist-test-slave-vr57 krb5kdc[24085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:00:45 dist-test-slave-vr57 krb5kdc[24085](info): set up 2 sockets
Sep 23 18:00:45 dist-test-slave-vr57 krb5kdc[24085](info): commencing operation
krb5kdc: starting...
W20260923 18:00:46.429325 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:00:46 dist-test-slave-vr57 krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:00:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:00:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:00:46.554752 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:42255
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:42255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:46.673343 24098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:46.673650 24098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:46.673691 24098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:46.677891 24098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:46.677963 24098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:46.677983 24098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:46.678000 24098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:46.678015 24098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:46.683449 24098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:42255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:42255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:46.685041 24098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:46.686143 24098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:46.695041 24103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:46.695063 24106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:46.695053 24104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:46.695399 24098 server_base.cc:1034] running on GCE node
I20260923 18:00:46.696071 24098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:46.697234 24098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:46.698431 24098 hybrid_clock.cc:648] HybridClock initialized: now 1790186446698411 us; error 37 us; skew 500 ppm
Sep 23 18:00:46 dist-test-slave-vr57 krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186446, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:46.701298 24098 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:00:46.702204 24098 webserver.cc:460] Webserver started at http://127.15.139.190:36723/ using document root <none> and password file <none>
I20260923 18:00:46.702737 24098 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:46.702795 24098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:46.702961 24098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:46.705127 24098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "d9cbe3f92dbe401fa38dd83147108482"
format_stamp: "Formatted at 2026-09-23 18:00:46 on dist-test-slave-vr57"
server_key: "e3f240797bb07d2ef57a7a824507ec33"
server_key_iv: "139ea00c37109514e42f134e342aae41"
server_key_version: "encryptionkey@0"
I20260923 18:00:46.705662 24098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "d9cbe3f92dbe401fa38dd83147108482"
format_stamp: "Formatted at 2026-09-23 18:00:46 on dist-test-slave-vr57"
server_key: "e3f240797bb07d2ef57a7a824507ec33"
server_key_iv: "139ea00c37109514e42f134e342aae41"
server_key_version: "encryptionkey@0"
I20260923 18:00:46.709525 24098 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:00:46.711982 24113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:46.713276 24098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:00:46.713376 24098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "d9cbe3f92dbe401fa38dd83147108482"
format_stamp: "Formatted at 2026-09-23 18:00:46 on dist-test-slave-vr57"
server_key: "e3f240797bb07d2ef57a7a824507ec33"
server_key_iv: "139ea00c37109514e42f134e342aae41"
server_key_version: "encryptionkey@0"
I20260923 18:00:46.713464 24098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:46.735589 24098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:46.739140 24098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:46.739310 24098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:46.746763 24098 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:42255
I20260923 18:00:46.746776 24165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:42255 every 8 connection(s)
I20260923 18:00:46.747815 24098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:46.750712 24166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:00:46.751835 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 24098
I20260923 18:00:46.751981 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:00:46.752327 15918 external_mini_cluster.cc:1428] Setting key c9d86a53519a5704df5050a86f2dc619
I20260923 18:00:46.757182 24166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: Bootstrap starting.
I20260923 18:00:46.759538 24166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: Neither blocks nor log segments found. Creating new log.
I20260923 18:00:46.760299 24166 log.cc:824] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: Log is configured to *not* fsync() on all Append() calls
Sep 23 18:00:46 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:46.762862 24166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: No bootstrap required, opened a new log
I20260923 18:00:46.767882 24166 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbe3f92dbe401fa38dd83147108482" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42255 } }
I20260923 18:00:46.768194 24166 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:00:46.768256 24166 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cbe3f92dbe401fa38dd83147108482, State: Initialized, Role: FOLLOWER
I20260923 18:00:46.768335 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:46.753876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56392 (local address 127.15.139.190:42255)
0923 18:00:46.754357 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:46.754372 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:46.754408 (+ 36us) server_negotiation.cc:407] Connection header received
0923 18:00:46.755066 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:46.755088 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:46.755438 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:46.755889 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:46.756714 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:46.757765 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:46.758640 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:46.759135 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:46.759306 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:46.762432 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:46.762437 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:46.762452 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:46.762480 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:46.764774 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:46.765313 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:46.765321 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:46.765329 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:46.765551 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:46.765906 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:46.765909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:46.765911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:46.766465 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:46.766700 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:46.766916 (+ 216us) server_negotiation.cc:299] Negotiation successful
0923 18:00:46.767257 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":146,"threads_started":1}
I20260923 18:00:46.768784 24166 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbe3f92dbe401fa38dd83147108482" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42255 } }
I20260923 18:00:46.768945 24166 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:46.769024 24166 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:46.769165 24166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:00:46.770179 24166 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbe3f92dbe401fa38dd83147108482" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42255 } }
I20260923 18:00:46.770535 24166 leader_election.cc:302] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9cbe3f92dbe401fa38dd83147108482; no voters:
I20260923 18:00:46.770907 24166 leader_election.cc:288] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:00:46.771013 24171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:00:46.771252 24171 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 1 LEADER]: Becoming Leader. State: Replica: d9cbe3f92dbe401fa38dd83147108482, State: Running, Role: LEADER
I20260923 18:00:46.771610 24171 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbe3f92dbe401fa38dd83147108482" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42255 } }
I20260923 18:00:46.772353 24166 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:46.773290 24173 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9cbe3f92dbe401fa38dd83147108482. Latest consensus state: current_term: 1 leader_uuid: "d9cbe3f92dbe401fa38dd83147108482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbe3f92dbe401fa38dd83147108482" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42255 } } }
I20260923 18:00:46.773336 24172 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9cbe3f92dbe401fa38dd83147108482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbe3f92dbe401fa38dd83147108482" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42255 } } }
I20260923 18:00:46.773432 24172 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:46.773432 24173 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [sys.catalog]: This master's current role is: LEADER
W20260923 18:00:46.777485 24187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:00:46.777576 24187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:00:46.777649 24180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:46.778330 24180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:46.784194 24180 catalog_manager.cc:1347] Generated new cluster ID: 91ebef71a01348f48b0e07527197541b
I20260923 18:00:46.784276 24180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:46.801993 24180 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:00:46.803375 24180 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:46.810428 24180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: Generated new TSK 0
I20260923 18:00:46.811373 24180 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:00:46.847584 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42255
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:46.964259 24192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:46.964581 24192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:46.964655 24192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:46.968891 24192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:46.968987 24192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:46.969091 24192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:46.974931 24192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:46.976210 24192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:46.977335 24192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:46.989529 24197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:46.989531 24200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:46.989609 24192 server_base.cc:1034] running on GCE node
W20260923 18:00:46.989835 24198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:46.990221 24192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:46.990978 24192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:46.992168 24192 hybrid_clock.cc:648] HybridClock initialized: now 1790186446992131 us; error 53 us; skew 500 ppm
Sep 23 18:00:46 dist-test-slave-vr57 krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186446, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:46.995026 24192 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:00:46.995978 24192 webserver.cc:460] Webserver started at http://127.15.139.129:39703/ using document root <none> and password file <none>
I20260923 18:00:46.996546 24192 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:46.996652 24192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:46.996883 24192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:46.998937 24192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "e913c474f8934bf1b09a7605f67bfba3"
format_stamp: "Formatted at 2026-09-23 18:00:46 on dist-test-slave-vr57"
server_key: "1c998f06204e3094d69c0827e7f024ca"
server_key_iv: "c49954d31ebea8da0bc84a6fec75ef33"
server_key_version: "encryptionkey@0"
I20260923 18:00:46.999567 24192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "e913c474f8934bf1b09a7605f67bfba3"
format_stamp: "Formatted at 2026-09-23 18:00:46 on dist-test-slave-vr57"
server_key: "1c998f06204e3094d69c0827e7f024ca"
server_key_iv: "c49954d31ebea8da0bc84a6fec75ef33"
server_key_version: "encryptionkey@0"
I20260923 18:00:47.003382 24192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260923 18:00:47.006013 24207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:47.007407 24192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:47.007604 24192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "e913c474f8934bf1b09a7605f67bfba3"
format_stamp: "Formatted at 2026-09-23 18:00:46 on dist-test-slave-vr57"
server_key: "1c998f06204e3094d69c0827e7f024ca"
server_key_iv: "c49954d31ebea8da0bc84a6fec75ef33"
server_key_version: "encryptionkey@0"
I20260923 18:00:47.007740 24192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:47.025723 24192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:47.029031 24192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:47.029206 24192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:47.029767 24192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:47.030580 24192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:47.030628 24192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:47.030669 24192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:47.030685 24192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:47.039705 24192 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:45463
I20260923 18:00:47.039733 24320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:45463 every 8 connection(s)
I20260923 18:00:47.040683 24192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:47.044783 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 24192
I20260923 18:00:47.044914 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:00:47.045252 15918 external_mini_cluster.cc:1428] Setting key 36b3a52c0a641abefcb6220dcdda0ee0
Sep 23 18:00:47 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186446, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
I20260923 18:00:47.056761 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:47.042740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:47185 (local address 127.15.139.190:42255)
0923 18:00:47.042882 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:47.042891 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:47.044075 (+ 1184us) server_negotiation.cc:407] Connection header received
0923 18:00:47.045318 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:47.045324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:47.045391 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:47.045469 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:47.046996 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:47.047594 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:47.049139 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:47.049377 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:47.049487 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:47.052556 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:47.052561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:47.052568 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:47.052607 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:47.054416 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:47.054956 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:47.054960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:47.054962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:47.055058 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:47.055389 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:47.055394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:47.055397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:47.055703 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:47.055791 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:47.056390 (+ 599us) server_negotiation.cc:299] Negotiation successful
0923 18:00:47.056577 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:00:47.057684 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:47.043146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:47185)
0923 18:00:47.043917 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:47.043961 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:47.045022 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:47.045699 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:47.045707 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:47.046177 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:47.046818 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:47.046839 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:47.047717 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:47.047723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:47.049011 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:47.049020 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:47.049527 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:47.049531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:47.049708 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:47.050393 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:47.050410 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:47.052359 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:47.054547 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:47.054553 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:47.054567 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:47.054829 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:47.055152 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:47.055154 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:47.055155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:47.055287 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:47.055800 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:47.055806 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:47.056115 (+ 309us) client_negotiation.cc:769] Sending connection context
0923 18:00:47.056348 (+ 233us) client_negotiation.cc:240] Negotiation successful
0923 18:00:47.056698 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":112,"threads_started":1}
I20260923 18:00:47.058979 24321 heartbeater.cc:347] Connected to a master server at 127.15.139.190:42255
I20260923 18:00:47.059223 24321 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:47.059830 24321 heartbeater.cc:511] Master 127.15.139.190:42255 requested a full tablet report, sending...
I20260923 18:00:47.061537 24130 ts_manager.cc:194] Registered new tserver with Master: e913c474f8934bf1b09a7605f67bfba3 (127.15.139.129:45463)
I20260923 18:00:47.063289 24130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:47185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:00:47.075546 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42255
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:47.194142 24326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:47.194440 24326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:47.194519 24326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:47.198573 24326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:47.198663 24326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:47.198752 24326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:47.204039 24326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:47.205118 24326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:47.206148 24326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:47.215883 24331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:47.215814 24332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:47.215942 24334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:47.216173 24326 server_base.cc:1034] running on GCE node
I20260923 18:00:47.216653 24326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:47.217316 24326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:47.218506 24326 hybrid_clock.cc:648] HybridClock initialized: now 1790186447218485 us; error 31 us; skew 500 ppm
Sep 23 18:00:47 dist-test-slave-vr57 krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186447, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:47.221205 24326 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:00:47.222035 24326 webserver.cc:460] Webserver started at http://127.15.139.130:36539/ using document root <none> and password file <none>
I20260923 18:00:47.222594 24326 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:47.222653 24326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:47.222872 24326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:47.225103 24326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "aa08312c583f4f00b849b43570a9e7eb"
format_stamp: "Formatted at 2026-09-23 18:00:47 on dist-test-slave-vr57"
server_key: "e7cd05e10b5159bbb94f98b3c736195c"
server_key_iv: "ca1475b8767285a5d3c1cdff9120878b"
server_key_version: "encryptionkey@0"
I20260923 18:00:47.225715 24326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "aa08312c583f4f00b849b43570a9e7eb"
format_stamp: "Formatted at 2026-09-23 18:00:47 on dist-test-slave-vr57"
server_key: "e7cd05e10b5159bbb94f98b3c736195c"
server_key_iv: "ca1475b8767285a5d3c1cdff9120878b"
server_key_version: "encryptionkey@0"
I20260923 18:00:47.229661 24326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260923 18:00:47.232223 24341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:47.233582 24326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:00:47.233708 24326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "aa08312c583f4f00b849b43570a9e7eb"
format_stamp: "Formatted at 2026-09-23 18:00:47 on dist-test-slave-vr57"
server_key: "e7cd05e10b5159bbb94f98b3c736195c"
server_key_iv: "ca1475b8767285a5d3c1cdff9120878b"
server_key_version: "encryptionkey@0"
I20260923 18:00:47.233820 24326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:47.244941 24326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:47.249058 24326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:47.249294 24326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:47.249980 24326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:47.250916 24326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:47.250975 24326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:47.251049 24326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:47.251086 24326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:47.260845 24326 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:44383
I20260923 18:00:47.260875 24454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:44383 every 8 connection(s)
I20260923 18:00:47.262003 24326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:47.262997 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 24326
I20260923 18:00:47.263109 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:00:47.263412 15918 external_mini_cluster.cc:1428] Setting key cde72fcb217b73919365b299ed1c3376
Sep 23 18:00:47 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186447, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
I20260923 18:00:47.277072 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:47.264208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:41269 (local address 127.15.139.190:42255)
0923 18:00:47.264337 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:47.264342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:47.265641 (+ 1299us) server_negotiation.cc:407] Connection header received
0923 18:00:47.266583 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:47.266586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:47.266643 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:47.266733 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:47.268022 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:47.268585 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:47.269765 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:47.270007 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:47.270116 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:47.272981 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:47.272984 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:47.272987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:47.273034 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:47.274677 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:47.275373 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:47.275376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:47.275377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:47.275461 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:47.275933 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:47.275936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:47.275938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:47.276158 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:47.276271 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:47.276778 (+ 507us) server_negotiation.cc:299] Negotiation successful
0923 18:00:47.276904 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 18:00:47.277987 24457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:47.264746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:41269)
0923 18:00:47.265492 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:47.265536 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:47.266361 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:47.266911 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:47.266919 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:47.267282 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:47.267872 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:47.267887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:47.268715 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:47.268719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:47.269639 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:47.269648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:47.270160 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:47.270164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:47.270334 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:47.271001 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:47.271018 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:47.272807 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:47.274786 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:47.274791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:47.274804 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:47.275265 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:47.275641 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:47.275646 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:47.275648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:47.275791 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:47.276269 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:47.276277 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:47.276537 (+ 260us) client_negotiation.cc:769] Sending connection context
0923 18:00:47.276726 (+ 189us) client_negotiation.cc:240] Negotiation successful
0923 18:00:47.277086 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":121,"threads_started":1}
I20260923 18:00:47.279281 24455 heartbeater.cc:347] Connected to a master server at 127.15.139.190:42255
I20260923 18:00:47.279595 24455 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:47.280119 24455 heartbeater.cc:511] Master 127.15.139.190:42255 requested a full tablet report, sending...
I20260923 18:00:47.281250 24130 ts_manager.cc:194] Registered new tserver with Master: aa08312c583f4f00b849b43570a9e7eb (127.15.139.130:44383)
I20260923 18:00:47.281841 24130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:41269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:00:47.294803 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42255
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:47.417030 24460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:47.417310 24460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:47.417356 24460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:47.421947 24460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:47.422024 24460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:47.422117 24460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:47.427660 24460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:47.428755 24460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:47.429782 24460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:47.439972 24465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:47.439966 24468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:47.440038 24466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:47.440227 24460 server_base.cc:1034] running on GCE node
I20260923 18:00:47.440608 24460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:47.441326 24460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:47.442482 24460 hybrid_clock.cc:648] HybridClock initialized: now 1790186447442460 us; error 35 us; skew 500 ppm
Sep 23 18:00:47 dist-test-slave-vr57 krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186447, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:47.445338 24460 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:00:47.446239 24460 webserver.cc:460] Webserver started at http://127.15.139.131:46641/ using document root <none> and password file <none>
I20260923 18:00:47.446828 24460 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:47.446920 24460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:47.447140 24460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:00:47.449239 24460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "0d336f776d6f4dad98172f564ff30497"
format_stamp: "Formatted at 2026-09-23 18:00:47 on dist-test-slave-vr57"
server_key: "72fbab69189f41bd9bcefdc8e0d409f2"
server_key_iv: "34327c69e057998322dcd436a3fd864d"
server_key_version: "encryptionkey@0"
I20260923 18:00:47.449837 24460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "0d336f776d6f4dad98172f564ff30497"
format_stamp: "Formatted at 2026-09-23 18:00:47 on dist-test-slave-vr57"
server_key: "72fbab69189f41bd9bcefdc8e0d409f2"
server_key_iv: "34327c69e057998322dcd436a3fd864d"
server_key_version: "encryptionkey@0"
I20260923 18:00:47.454134 24460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260923 18:00:47.456705 24475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:47.458022 24460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:00:47.458156 24460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "0d336f776d6f4dad98172f564ff30497"
format_stamp: "Formatted at 2026-09-23 18:00:47 on dist-test-slave-vr57"
server_key: "72fbab69189f41bd9bcefdc8e0d409f2"
server_key_iv: "34327c69e057998322dcd436a3fd864d"
server_key_version: "encryptionkey@0"
I20260923 18:00:47.458272 24460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:47.469158 24460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:47.472549 24460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:47.472757 24460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:47.473367 24460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:47.474283 24460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:47.474349 24460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:47.474417 24460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:47.474463 24460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:47.483852 24460 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:40271
I20260923 18:00:47.483870 24588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:40271 every 8 connection(s)
I20260923 18:00:47.484871 24460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:47.492112 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 24460
I20260923 18:00:47.492280 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:00:47.492561 15918 external_mini_cluster.cc:1428] Setting key 58d1814332b56b97b1e4d7e2cafe23d8
Sep 23 18:00:47 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186447, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:00:47.500382 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:47.486879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:46011 (local address 127.15.139.190:42255)
0923 18:00:47.487044 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:47.487048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:47.488223 (+ 1175us) server_negotiation.cc:407] Connection header received
0923 18:00:47.489206 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:47.489210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:47.489270 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:47.489351 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:47.490713 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:47.491263 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:47.492974 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:47.493317 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:47.493436 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:47.496459 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:47.496463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:47.496465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:47.496492 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:47.498014 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:47.498582 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:47.498586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:47.498588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:47.498677 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:47.499060 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:47.499064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:47.499066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:47.499265 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:47.499383 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:47.500067 (+ 684us) server_negotiation.cc:299] Negotiation successful
0923 18:00:47.500196 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260923 18:00:47.501231 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:47.487353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:46011)
0923 18:00:47.488064 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:47.488109 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:47.488977 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:47.489529 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:47.489536 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:47.489900 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:47.490559 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:47.490582 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:47.491401 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:47.491404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:47.492792 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:47.492804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:47.493494 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:47.493499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:47.493739 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:47.494502 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:47.494520 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:47.496239 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:00:47.498158 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:47.498165 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:47.498184 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:47.498467 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:47.498782 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:00:47.498785 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:00:47.498788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:00:47.498952 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:00:47.499394 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:00:47.499404 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:00:47.499770 (+ 366us) client_negotiation.cc:769] Sending connection context
0923 18:00:47.500007 (+ 237us) client_negotiation.cc:240] Negotiation successful
0923 18:00:47.500367 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":175,"threads_started":1}
I20260923 18:00:47.502342 24589 heartbeater.cc:347] Connected to a master server at 127.15.139.190:42255
I20260923 18:00:47.502578 24589 heartbeater.cc:464] Registering TS with master...
I20260923 18:00:47.503094 24589 heartbeater.cc:511] Master 127.15.139.190:42255 requested a full tablet report, sending...
I20260923 18:00:47.504264 24130 ts_manager.cc:194] Registered new tserver with Master: 0d336f776d6f4dad98172f564ff30497 (127.15.139.131:40271)
I20260923 18:00:47.504833 24130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:46011
I20260923 18:00:47.506588 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab.
Entry for principal oryx/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab.
WARNING: no policy specified for oryx/127.15.139.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab.
Entry for principal oryx/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab.
WARNING: no policy specified for oryx/127.15.139.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.139.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab.
Entry for principal oryx/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab.
I20260923 18:00:47.582293 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 24192
I20260923 18:00:47.588331 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 24326
I20260923 18:00:47.594092 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 24460
I20260923 18:00:47.599673 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 24098
I20260923 18:00:47.606124 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:42255
--webserver_interface=127.15.139.190
--webserver_port=36723
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:42255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:00:47.722380 24598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:47.722671 24598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:47.722716 24598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:47.726837 24598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:00:47.726922 24598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:47.726945 24598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:47.726966 24598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:00:47.726984 24598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:00:47.732394 24598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:42255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:42255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=36723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:47.733450 24598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:47.734452 24598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:47.743337 24604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:47.743345 24606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:47.743402 24603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:47.743824 24598 server_base.cc:1034] running on GCE node
I20260923 18:00:47.744498 24598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:47.745697 24598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:47.746861 24598 hybrid_clock.cc:648] HybridClock initialized: now 1790186447746835 us; error 41 us; skew 500 ppm
Sep 23 18:00:47 dist-test-slave-vr57 krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186447, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:47.749714 24598 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:00:47.750702 24598 webserver.cc:460] Webserver started at http://127.15.139.190:36723/ using document root <none> and password file <none>
I20260923 18:00:47.751292 24598 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:47.751387 24598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:47.755342 24598 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:47.757558 24613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:47.758843 24598 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:47.759063 24598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "d9cbe3f92dbe401fa38dd83147108482"
format_stamp: "Formatted at 2026-09-23 18:00:46 on dist-test-slave-vr57"
server_key: "e3f240797bb07d2ef57a7a824507ec33"
server_key_iv: "139ea00c37109514e42f134e342aae41"
server_key_version: "encryptionkey@0"
I20260923 18:00:47.759696 24598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:47.769795 24598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:47.773030 24598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:47.773226 24598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:47.780632 24598 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:42255
I20260923 18:00:47.780646 24665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:42255 every 8 connection(s)
I20260923 18:00:47.781704 24598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:00:47.782845 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 24598
I20260923 18:00:47.783349 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:45463
--local_ip_for_outbound_sockets=127.15.139.129
--tserver_master_addrs=127.15.139.190:42255
--webserver_port=39703
--webserver_interface=127.15.139.129
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.139.129
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:00:47.787029 24666 sys_catalog.cc:263] Verifying existing consensus state
I20260923 18:00:47.788757 24666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: Bootstrap starting.
I20260923 18:00:47.805720 24666 log.cc:824] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: Log is configured to *not* fsync() on all Append() calls
I20260923 18:00:47.812711 24666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 18:00:47.813110 24666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: Bootstrap complete.
I20260923 18:00:47.817574 24666 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbe3f92dbe401fa38dd83147108482" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42255 } }
I20260923 18:00:47.818044 24666 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cbe3f92dbe401fa38dd83147108482, State: Initialized, Role: FOLLOWER
I20260923 18:00:47.818607 24666 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbe3f92dbe401fa38dd83147108482" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42255 } }
I20260923 18:00:47.818739 24666 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:00:47.818825 24666 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:00:47.818930 24666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 1 FOLLOWER]: Advancing to term 2
I20260923 18:00:47.821077 24666 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbe3f92dbe401fa38dd83147108482" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42255 } }
I20260923 18:00:47.821432 24666 leader_election.cc:302] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9cbe3f92dbe401fa38dd83147108482; no voters:
I20260923 18:00:47.821810 24666 leader_election.cc:288] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 18:00:47.821904 24670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 18:00:47.822149 24670 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [term 2 LEADER]: Becoming Leader. State: Replica: d9cbe3f92dbe401fa38dd83147108482, State: Running, Role: LEADER
I20260923 18:00:47.822480 24670 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbe3f92dbe401fa38dd83147108482" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42255 } }
I20260923 18:00:47.823205 24666 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:00:47.824029 24671 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d9cbe3f92dbe401fa38dd83147108482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbe3f92dbe401fa38dd83147108482" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42255 } } }
I20260923 18:00:47.824080 24672 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9cbe3f92dbe401fa38dd83147108482. Latest consensus state: current_term: 2 leader_uuid: "d9cbe3f92dbe401fa38dd83147108482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cbe3f92dbe401fa38dd83147108482" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42255 } } }
I20260923 18:00:47.824168 24671 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:47.824200 24672 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482 [sys.catalog]: This master's current role is: LEADER
I20260923 18:00:47.829217 24685 catalog_manager.cc:1259] Loaded cluster ID: 91ebef71a01348f48b0e07527197541b
I20260923 18:00:47.829315 24685 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: loading cluster ID for follower catalog manager: success
I20260923 18:00:47.831928 24685 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: acquiring CA information for follower catalog manager: success
I20260923 18:00:47.833416 24685 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 18:00:47.833561 24678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:00:47.835043 24678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:00:47.835561 24678 catalog_manager.cc:1259] Loaded cluster ID: 91ebef71a01348f48b0e07527197541b
I20260923 18:00:47.835630 24678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:00:47.836277 24678 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:00:47.836735 24678 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d9cbe3f92dbe401fa38dd83147108482: Loaded TSK: 0
I20260923 18:00:47.837525 24678 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 18:00:47.904680 24668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:47.904965 24668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:47.905041 24668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:47.909282 24668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:47.909363 24668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:47.909464 24668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:00:47.915162 24668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.129.keytab
--principal=oryx/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:45463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=39703
--tserver_master_addrs=127.15.139.190:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:47.916388 24668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:47.917563 24668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:47.927847 24693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:47.927932 24695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:47.927847 24692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:47.928184 24668 server_base.cc:1034] running on GCE node
I20260923 18:00:47.928689 24668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:47.929394 24668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:47.930615 24668 hybrid_clock.cc:648] HybridClock initialized: now 1790186447930578 us; error 53 us; skew 500 ppm
Sep 23 18:00:47 dist-test-slave-vr57 krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186447, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:47.941104 24668 init.cc:377] Logged in from keytab as oryx/127.15.139.129@KRBTEST.COM (short username oryx)
I20260923 18:00:47.942169 24668 webserver.cc:460] Webserver started at http://127.15.139.129:39703/ using document root <none> and password file <none>
I20260923 18:00:47.942719 24668 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:47.942795 24668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:47.947039 24668 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:00:47.949487 24702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:47.951392 24668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:00:47.951629 24668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "e913c474f8934bf1b09a7605f67bfba3"
format_stamp: "Formatted at 2026-09-23 18:00:46 on dist-test-slave-vr57"
server_key: "1c998f06204e3094d69c0827e7f024ca"
server_key_iv: "c49954d31ebea8da0bc84a6fec75ef33"
server_key_version: "encryptionkey@0"
I20260923 18:00:47.952382 24668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:48.001308 24668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:48.004532 24668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:48.004729 24668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:48.005952 24668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:48.007053 24668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:48.007112 24668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:48.007160 24668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:48.007192 24668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:48.017305 24668 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:45463
I20260923 18:00:48.017340 24815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:45463 every 8 connection(s)
I20260923 18:00:48.018349 24668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:00:48.021267 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 24668
I20260923 18:00:48.021775 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:44383
--local_ip_for_outbound_sockets=127.15.139.130
--tserver_master_addrs=127.15.139.190:42255
--webserver_port=36539
--webserver_interface=127.15.139.130
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.139.130
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 18:00:48.034761 24819 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:48.020712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:51821 (local address 127.15.139.190:42255)
0923 18:00:48.021462 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:48.021494 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:48.021758 (+ 264us) server_negotiation.cc:407] Connection header received
0923 18:00:48.023303 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:48.023314 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:48.023729 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:48.024145 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:48.025471 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.026549 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.028598 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.029120 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.029267 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.034005 (+ 4738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:51821: BlockingRecv error: recv got EOF from 127.15.139.129:51821 (error 108)
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":203,"threads_started":1}
I20260923 18:00:48.034874 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:48.020713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:51821)
0923 18:00:48.021580 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:48.021632 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:48.022766 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:48.024217 (+ 1451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:48.024226 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:48.024745 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:48.025222 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.025238 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.026767 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.026775 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.028444 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.028461 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.029313 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.029319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.029586 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.030312 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:48.030341 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:48.033771 (+ 3430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":165,"threads_started":1}
W20260923 18:00:48.035588 24816 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:42255 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.139.190:42255: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:48.042125 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:48.036041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:46309)
0923 18:00:48.036254 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:48.036268 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:48.036338 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:48.036615 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:48.036617 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:48.036788 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:48.036917 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.036923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.037981 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.037985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.038632 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.038643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.039264 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.039267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.039351 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.040000 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:48.040014 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:48.041984 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129}
W20260923 18:00:48.042390 24819 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:48.036164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:46309 (local address 127.15.139.190:42255)
0923 18:00:48.036295 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:48.036300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:48.036318 (+ 18us) server_negotiation.cc:407] Connection header received
0923 18:00:48.036432 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:48.036435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:48.036488 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:48.036580 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:48.037027 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.037862 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.038748 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.039119 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.039232 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.042130 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:46309: BlockingRecv error: recv got EOF from 127.15.139.129:46309 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:48.048702 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:48.042895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:60239)
0923 18:00:48.043026 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:48.043042 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:48.043125 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:48.043396 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:48.043408 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:48.043621 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:48.043718 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.043726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.044603 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.044607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.045491 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.045504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.045978 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.045984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.046077 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.046569 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:48.046590 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:48.048571 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260923 18:00:48.048915 24819 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:48.043080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:60239 (local address 127.15.139.190:42255)
0923 18:00:48.043201 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:48.043205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:48.043219 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:00:48.043252 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:48.043254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:48.043291 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:48.043362 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:48.043858 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.044490 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.045635 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.045862 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.045950 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.048682 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:60239: BlockingRecv error: recv got EOF from 127.15.139.129:60239 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260923 18:00:48.049016 24816 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 18:00:48.143172 24820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:48.143473 24820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:48.143586 24820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:48.147794 24820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:48.147867 24820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:48.147965 24820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:00:48.153626 24820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.130.keytab
--principal=oryx/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:44383
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=36539
--tserver_master_addrs=127.15.139.190:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:48.154771 24820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:48.155885 24820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:48.165784 24826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:48.165779 24828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:48.165823 24825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:48.166275 24820 server_base.cc:1034] running on GCE node
I20260923 18:00:48.166713 24820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:48.167402 24820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:48.168623 24820 hybrid_clock.cc:648] HybridClock initialized: now 1790186448168584 us; error 49 us; skew 500 ppm
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186448, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:48.171672 24820 init.cc:377] Logged in from keytab as oryx/127.15.139.130@KRBTEST.COM (short username oryx)
I20260923 18:00:48.172533 24820 webserver.cc:460] Webserver started at http://127.15.139.130:36539/ using document root <none> and password file <none>
I20260923 18:00:48.173180 24820 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:48.173259 24820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:48.177431 24820 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:00:48.179615 24835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:48.180915 24820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:00:48.181031 24820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "aa08312c583f4f00b849b43570a9e7eb"
format_stamp: "Formatted at 2026-09-23 18:00:47 on dist-test-slave-vr57"
server_key: "e7cd05e10b5159bbb94f98b3c736195c"
server_key_iv: "ca1475b8767285a5d3c1cdff9120878b"
server_key_version: "encryptionkey@0"
I20260923 18:00:48.181620 24820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:48.216902 24820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:48.220283 24820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:48.220443 24820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:48.220981 24820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:48.221757 24820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:48.221800 24820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:48.221840 24820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:48.221861 24820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:48.231069 24820 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:44383
I20260923 18:00:48.231091 24948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:44383 every 8 connection(s)
I20260923 18:00:48.232136 24820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:00:48.240548 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 24820
I20260923 18:00:48.241043 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:40271
--local_ip_for_outbound_sockets=127.15.139.131
--tserver_master_addrs=127.15.139.190:42255
--webserver_port=46641
--webserver_interface=127.15.139.131
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.139.131
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 18:00:48.243944 24819 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:48.233909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:50263 (local address 127.15.139.190:42255)
0923 18:00:48.234073 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:48.234078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:48.235216 (+ 1138us) server_negotiation.cc:407] Connection header received
0923 18:00:48.236170 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:48.236174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:48.236235 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:48.236326 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:48.237561 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.238158 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.239303 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.239625 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.239723 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.243835 (+ 4112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:50263: BlockingRecv error: recv got EOF from 127.15.139.130:50263 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 18:00:48.244786 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:48.234415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:50263)
0923 18:00:48.235061 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:48.235104 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:48.235939 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:48.236496 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:48.236503 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:48.236883 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:48.237379 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.237393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.238287 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.238289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.239174 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.239183 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.239761 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.239764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.239909 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.240667 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:48.240692 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:48.243709 (+ 3017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":119,"threads_started":1}
W20260923 18:00:48.245311 24949 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:42255 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.139.190:42255: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:48.250944 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:48.245749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:54863)
0923 18:00:48.245846 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:48.245862 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:48.245968 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:48.246194 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:48.246197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:48.246356 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:48.246454 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.246459 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.247242 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.247247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.247844 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.247853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.248299 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.248302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.248393 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.248884 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:48.248895 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:48.250801 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 18:00:48.251039 24819 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:48.245851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:54863 (local address 127.15.139.190:42255)
0923 18:00:48.245976 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:48.245980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:48.245996 (+ 16us) server_negotiation.cc:407] Connection header received
0923 18:00:48.246047 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:48.246050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:48.246095 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:48.246180 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:48.246565 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.247127 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.247981 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.248195 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.248300 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.250922 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:54863: BlockingRecv error: recv got EOF from 127.15.139.130:54863 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:48.256836 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:48.251634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:59989)
0923 18:00:48.251727 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:48.251741 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:48.251812 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:48.252086 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:48.252089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:48.252210 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:48.252280 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.252284 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.253277 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.253279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.253790 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.253797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.254343 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.254345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.254403 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.254885 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:48.254896 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:48.256709 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 18:00:48.257038 24949 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 18:00:48.257067 24819 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:48.251714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:59989 (local address 127.15.139.190:42255)
0923 18:00:48.251851 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:48.251857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:48.251874 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:00:48.251919 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:48.251925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:48.251975 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:48.252057 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:48.252375 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.253177 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.253888 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.254230 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.254321 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.256825 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:59989: BlockingRecv error: recv got EOF from 127.15.139.130:59989 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260923 18:00:48.356590 24952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:00:48.356848 24952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:00:48.356891 24952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:00:48.361021 24952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:00:48.361074 24952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:00:48.361157 24952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:00:48.366410 24952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/krb5kdc/oryx_127.15.139.131.keytab
--principal=oryx/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:40271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=46641
--tserver_master_addrs=127.15.139.190:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:00:48.367442 24952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:00:48.368525 24952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:00:48.378217 24958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:48.378217 24957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:00:48.378235 24960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:00:48.378562 24952 server_base.cc:1034] running on GCE node
I20260923 18:00:48.379010 24952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:00:48.379822 24952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:00:48.381009 24952 hybrid_clock.cc:648] HybridClock initialized: now 1790186448380972 us; error 44 us; skew 500 ppm
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186448, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:00:48.384130 24952 init.cc:377] Logged in from keytab as oryx/127.15.139.131@KRBTEST.COM (short username oryx)
I20260923 18:00:48.385028 24952 webserver.cc:460] Webserver started at http://127.15.139.131:46641/ using document root <none> and password file <none>
I20260923 18:00:48.385663 24952 fs_manager.cc:359] Metadata directory not provided
I20260923 18:00:48.385740 24952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:00:48.390039 24952 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:00:48.392107 24967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:48.393401 24952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:00:48.393555 24952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "0d336f776d6f4dad98172f564ff30497"
format_stamp: "Formatted at 2026-09-23 18:00:47 on dist-test-slave-vr57"
server_key: "72fbab69189f41bd9bcefdc8e0d409f2"
server_key_iv: "34327c69e057998322dcd436a3fd864d"
server_key_version: "encryptionkey@0"
I20260923 18:00:48.394098 24952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:00:48.413211 24952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:00:48.416760 24952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:00:48.416972 24952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:00:48.417583 24952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:00:48.418552 24952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:00:48.418630 24952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:48.418702 24952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:00:48.418748 24952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:00:48.429222 24952 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:40271
I20260923 18:00:48.429250 25080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:40271 every 8 connection(s)
I20260923 18:00:48.430298 24952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:00:48.438608 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 24952
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 18:00:48.442420 24819 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:48.432308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:56641 (local address 127.15.139.190:42255)
0923 18:00:48.432479 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:48.432484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:48.433578 (+ 1094us) server_negotiation.cc:407] Connection header received
0923 18:00:48.434412 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:48.434415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:48.434462 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:48.434543 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:48.435798 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.436513 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.437791 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.438018 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.438111 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.442290 (+ 4179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:56641: BlockingRecv error: recv got EOF from 127.15.139.131:56641 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260923 18:00:48.443076 25083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:48.432714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:56641)
0923 18:00:48.433395 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:48.433456 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:48.434222 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:48.434711 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:48.434717 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:48.435067 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:48.435659 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.435675 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.436660 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.436663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.437667 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.437677 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.438181 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.438184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.438330 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.439137 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:48.439171 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:48.442159 (+ 2988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":116,"threads_started":1}
W20260923 18:00:48.443740 25081 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:42255 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.139.190:42255: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:48.449815 25083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:48.444241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:52695)
0923 18:00:48.444433 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:48.444448 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:48.444543 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:48.444856 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:48.444870 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:48.445027 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:48.445171 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.445177 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.445872 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.445878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.446425 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.446432 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.446866 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.446870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.446959 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.447408 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:48.447419 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:48.449647 (+ 2228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120}
W20260923 18:00:48.449887 24819 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:48.444340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:52695 (local address 127.15.139.190:42255)
0923 18:00:48.444452 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:48.444456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:48.444494 (+ 38us) server_negotiation.cc:407] Connection header received
0923 18:00:48.444692 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:48.444695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:48.444738 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:48.444817 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:48.445286 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.445771 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.446539 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.446761 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.446865 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.449779 (+ 2914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:52695: BlockingRecv error: recv got EOF from 127.15.139.131:52695 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 18:00:48.450121 25085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:48.439742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.15.139.190:42255)
0923 18:00:48.440035 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:48.440038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:48.440049 (+ 11us) server_negotiation.cc:407] Connection header received
0923 18:00:48.440080 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:48.440082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:48.440117 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:48.440191 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:48.441084 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.441910 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.443016 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.443338 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.443491 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.445195 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:00:48.445200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:00:48.445216 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:00:48.445242 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:00:48.447411 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:48.447902 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:48.447907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:48.447913 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:48.448178 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:00:48.448465 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:00:48.448469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:00:48.448470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:00:48.449235 (+ 765us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:00:48.449411 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0923 18:00:48.449733 (+ 322us) server_negotiation.cc:299] Negotiation successful
0923 18:00:48.449987 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":109,"threads_started":1}
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:48 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:48.455410 25083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:48.450522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:53927)
0923 18:00:48.450637 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:48.450651 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:48.450731 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:48.450951 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:48.450954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:48.451111 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:48.451197 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.451204 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.451968 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.451971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.452690 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:48.452700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.453274 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.453276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:48.453335 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.453712 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:48.453723 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:48.455316 (+ 1593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 18:00:48.455602 24819 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:48.450546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:53927 (local address 127.15.139.190:42255)
0923 18:00:48.450684 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:48.450689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:48.450702 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:00:48.450816 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:48.450819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:48.450855 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:48.450938 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:48.451313 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.451845 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.452811 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:48.453176 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:48.453266 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:48.455426 (+ 2160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:53927: BlockingRecv error: recv got EOF from 127.15.139.131:53927 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260923 18:00:48.455658 25081 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 23 18:00:49 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:49 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:49.055580 25086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:49.049753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:41137)
0923 18:00:49.050039 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:49.050053 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:49.050148 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:49.050440 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:49.050442 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:49.050628 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:49.050757 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:49.050763 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:49.051602 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:49.051605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:49.052246 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:49.052257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:49.052688 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:49.052692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:49.052797 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:49.053569 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:49.053580 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:49.055412 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":94,"threads_started":1}
W20260923 18:00:49.055758 25087 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:49.049889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:41137 (local address 127.15.139.190:42255)
0923 18:00:49.050160 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:49.050164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:49.050178 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:00:49.050218 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:49.050220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:49.050260 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:49.050348 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:49.050871 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:49.051422 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:49.052356 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:49.052578 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:49.052658 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:49.055594 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:41137: BlockingRecv error: recv got EOF from 127.15.139.129:41137 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":98,"threads_started":1}
Sep 23 18:00:49 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:49 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:49.263464 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:49.257625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:32885)
0923 18:00:49.257894 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:49.257908 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:49.258018 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:49.258335 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:49.258338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:49.258561 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:49.258656 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:49.258662 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:49.259418 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:49.259423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:49.260037 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:49.260047 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:49.260494 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:49.260497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:49.260603 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:49.261141 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:49.261153 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:49.263307 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":107,"threads_started":1}
W20260923 18:00:49.263734 25087 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:49.257695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:32885 (local address 127.15.139.190:42255)
0923 18:00:49.257817 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:49.257820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:49.257973 (+ 153us) server_negotiation.cc:407] Connection header received
0923 18:00:49.258131 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:49.258134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:49.258211 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:49.258311 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:49.258774 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:49.259309 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:49.260162 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:49.260384 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:49.260488 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:49.263474 (+ 2986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:32885: BlockingRecv error: recv got EOF from 127.15.139.130:32885 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 23 18:00:49 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:49 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:49.461958 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:49.456321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:40305)
0923 18:00:49.456604 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:49.456618 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:49.456720 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:49.456944 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:49.456945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:49.457134 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:49.457256 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:49.457262 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:49.458031 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:49.458034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:49.458692 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:49.458726 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:49.459183 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:49.459186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:49.459264 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:49.459829 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:49.459841 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:49.461849 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":71,"threads_started":1}
W20260923 18:00:49.462105 25087 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:49.456400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:40305 (local address 127.15.139.190:42255)
0923 18:00:49.456581 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:49.456586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:49.456686 (+ 100us) server_negotiation.cc:407] Connection header received
0923 18:00:49.456796 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:49.456798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:49.456843 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:49.456913 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:49.457351 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:49.457911 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:49.458827 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:49.459039 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:49.459150 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:49.461964 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:40305: BlockingRecv error: recv got EOF from 127.15.139.131:40305 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 23 18:00:50 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:50 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:50.062325 25090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:50.056549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:32979)
0923 18:00:50.056860 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:50.056877 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:50.056992 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:50.057271 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:50.057274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:50.057473 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:50.057567 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:50.057574 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:50.058415 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:50.058421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:50.058973 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:50.058981 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:50.059548 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:50.059552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:50.059641 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:50.060214 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:50.060227 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:50.062142 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":93,"threads_started":1}
W20260923 18:00:50.062381 25091 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:50.056604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:32979 (local address 127.15.139.190:42255)
0923 18:00:50.056906 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:50.056912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:50.056927 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:00:50.057094 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:50.057099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:50.057147 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:50.057242 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:50.057681 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:50.058298 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:50.059168 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:50.059405 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:50.059535 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:50.062258 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:32979: BlockingRecv error: recv got EOF from 127.15.139.129:32979 (error 108)
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":101,"threads_started":1}
Sep 23 18:00:50 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:50 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:50.270118 25092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:50.264415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:51843)
0923 18:00:50.264676 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:50.264693 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:50.264839 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:50.265180 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:50.265183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:50.265396 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:50.265500 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:50.265506 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:50.266282 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:50.266286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:50.266868 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:50.266876 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:50.267356 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:50.267360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:50.267447 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:50.268025 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:50.268040 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:50.269921 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":73,"threads_started":1}
W20260923 18:00:50.270284 25091 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:50.264545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:51843 (local address 127.15.139.190:42255)
0923 18:00:50.264677 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:50.264683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:50.264782 (+ 99us) server_negotiation.cc:407] Connection header received
0923 18:00:50.264945 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:50.264949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:50.265061 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:50.265154 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:50.265623 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:50.266166 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:50.267011 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:50.267240 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:50.267353 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:50.270148 (+ 2795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:51843: BlockingRecv error: recv got EOF from 127.15.139.130:51843 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 23 18:00:50 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:50 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:50.468875 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:50.462862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:39239)
0923 18:00:50.463161 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:50.463217 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:50.463344 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:50.463639 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:50.463641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:50.463846 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:50.463947 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:50.463953 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:50.464907 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:50.464913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:50.465496 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:50.465504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:50.466075 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:50.466079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:50.466166 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:50.466732 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:50.466744 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:50.468691 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":85,"threads_started":1}
W20260923 18:00:50.468936 25091 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:50.463087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:39239 (local address 127.15.139.190:42255)
0923 18:00:50.463231 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:50.463235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:50.463300 (+ 65us) server_negotiation.cc:407] Connection header received
0923 18:00:50.463433 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:50.463435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:50.463480 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:50.463574 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:50.464234 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:50.464775 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:50.465629 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:50.465856 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:50.466071 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:50.468827 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:39239: BlockingRecv error: recv got EOF from 127.15.139.131:39239 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 23 18:00:51 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:51 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:51.068821 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:51.063168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:40715)
0923 18:00:51.063609 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:51.063624 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:51.063739 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:51.064013 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:51.064016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:51.064198 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:51.064296 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:51.064302 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:51.065135 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:51.065137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:51.065726 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:51.065734 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:51.066229 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:51.066232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:51.066291 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:51.066827 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:51.066839 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:51.068694 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":98,"threads_started":1}
W20260923 18:00:51.068988 25095 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:51.063226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:40715 (local address 127.15.139.190:42255)
0923 18:00:51.063527 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:51.063532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:51.063703 (+ 171us) server_negotiation.cc:407] Connection header received
0923 18:00:51.063820 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:51.063823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:51.063874 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:51.063962 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:51.064413 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:51.065041 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:51.065835 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:51.066127 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:51.066215 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:51.068846 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:40715: BlockingRecv error: recv got EOF from 127.15.139.129:40715 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":63,"threads_started":1}
Sep 23 18:00:51 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:51 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:51.276806 25096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:51.271205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:59781)
0923 18:00:51.271467 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:51.271480 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:51.271676 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:51.271957 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:51.271960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:51.272188 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:51.272281 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:51.272287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:51.273164 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:51.273169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:51.273728 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:51.273735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:51.274209 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:51.274213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:51.274305 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:51.274832 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:51.274844 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:51.276694 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
W20260923 18:00:51.277119 25095 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:51.271238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:59781 (local address 127.15.139.190:42255)
0923 18:00:51.271402 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:51.271407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:51.271616 (+ 209us) server_negotiation.cc:407] Connection header received
0923 18:00:51.271787 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:51.271795 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:51.271845 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:51.271933 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:51.272397 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:51.273060 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:51.273844 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:51.274081 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:51.274183 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:51.276820 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:59781: BlockingRecv error: recv got EOF from 127.15.139.130:59781 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 23 18:00:51 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:51 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:51.476117 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:51.469749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:58389)
0923 18:00:51.470182 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:51.470197 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:51.470356 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:51.470708 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:51.470712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:51.470979 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:51.471087 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:51.471094 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:51.472015 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:51.472023 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:51.472693 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:51.472702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:51.473230 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:51.473234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:51.473336 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:51.473966 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:51.473980 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:51.475987 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":89,"threads_started":1}
W20260923 18:00:51.476279 25095 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:51.469864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:58389 (local address 127.15.139.190:42255)
0923 18:00:51.470063 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:51.470070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:51.470285 (+ 215us) server_negotiation.cc:407] Connection header received
0923 18:00:51.470526 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:51.470528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:51.470582 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:51.470687 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:51.471223 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:51.471889 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:51.472848 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:51.473103 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:51.473227 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:51.476148 (+ 2921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:58389: BlockingRecv error: recv got EOF from 127.15.139.131:58389 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 23 18:00:52 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:52 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:52.075804 25098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:52.069760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:37841)
0923 18:00:52.070028 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:52.070046 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:52.070162 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:52.070469 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:52.070471 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:52.070661 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:52.070772 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:52.070779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:52.071639 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:52.071642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:52.072266 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:52.072276 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:52.072741 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:52.072744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:52.072813 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:52.073455 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:52.073468 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:52.075633 (+ 2165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":98,"threads_started":1}
W20260923 18:00:52.075865 25099 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:52.069816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:37841 (local address 127.15.139.190:42255)
0923 18:00:52.070132 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:52.070135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:52.070155 (+ 20us) server_negotiation.cc:407] Connection header received
0923 18:00:52.070234 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:52.070236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:52.070278 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:52.070371 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:52.070892 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:52.071457 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:52.072396 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:52.072629 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:52.072714 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:52.075740 (+ 3026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:37841: BlockingRecv error: recv got EOF from 127.15.139.129:37841 (error 108)
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":170,"threads_started":1}
Sep 23 18:00:52 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:52 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:52.283447 25100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:52.277716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:56865)
0923 18:00:52.278066 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:52.278079 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:52.278201 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:52.278477 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:52.278480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:52.278721 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:52.278825 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:52.278831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:52.279615 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:52.279617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:52.280187 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:52.280194 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:52.280682 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:52.280684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:52.280744 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:52.281298 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:52.281309 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:52.283332 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":90,"threads_started":1}
W20260923 18:00:52.283681 25099 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:52.277801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:56865 (local address 127.15.139.190:42255)
0923 18:00:52.277963 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:52.277969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:52.278152 (+ 183us) server_negotiation.cc:407] Connection header received
0923 18:00:52.278301 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:52.278305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:52.278363 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:52.278450 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:52.278927 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:52.279534 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:52.280322 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:52.280570 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:52.280663 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:52.283547 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:56865: BlockingRecv error: recv got EOF from 127.15.139.130:56865 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 23 18:00:52 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:52 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:52.482759 25101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:52.476943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:33111)
0923 18:00:52.477377 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:52.477390 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:52.477521 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:52.477832 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:52.477838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:52.478027 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:52.478202 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:52.478214 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:52.479134 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:52.479136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:52.479761 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:52.479770 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:52.480215 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:52.480218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:52.480276 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:52.480780 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:52.480791 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:52.482647 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":88,"threads_started":1}
W20260923 18:00:52.483045 25099 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:52.477136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:33111 (local address 127.15.139.190:42255)
0923 18:00:52.477308 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:52.477312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:52.477467 (+ 155us) server_negotiation.cc:407] Connection header received
0923 18:00:52.477625 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:52.477627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:52.477703 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:52.477786 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:52.478318 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:52.479036 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:52.479878 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:52.480113 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:52.480197 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:52.482916 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:33111: BlockingRecv error: recv got EOF from 127.15.139.131:33111 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 23 18:00:53 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:53 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:53.089356 25102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:53.076831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:38951)
0923 18:00:53.077122 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:53.077139 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:53.077247 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:53.077615 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:53.077618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:53.077863 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:53.077981 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:53.077986 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:53.078781 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:53.078783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:53.079360 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:53.079367 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:53.079821 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:53.079824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:53.079886 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:53.087048 (+ 7162us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:53.087062 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:53.089193 (+ 2131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":91,"threads_started":1}
W20260923 18:00:53.089517 25103 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:53.076879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:38951 (local address 127.15.139.190:42255)
0923 18:00:53.077167 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:53.077173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:53.077196 (+ 23us) server_negotiation.cc:407] Connection header received
0923 18:00:53.077372 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:53.077377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:53.077436 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:53.077531 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:53.078125 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:53.078686 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:53.079461 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:53.079731 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:53.079812 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:53.089351 (+ 9539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:38951: BlockingRecv error: recv got EOF from 127.15.139.129:38951 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":86,"threads_started":1}
Sep 23 18:00:53 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:53 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:53.290750 25104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:53.284341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:33031)
0923 18:00:53.284624 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:53.284639 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:53.284726 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:53.285105 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:53.285108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:53.285315 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:53.285410 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:53.285416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:53.286484 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:53.286487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:53.287053 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:53.287061 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:53.287551 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:53.287556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:53.287640 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:53.288199 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:53.288210 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:53.290585 (+ 2375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":109,"threads_started":1}
W20260923 18:00:53.290786 25103 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:53.284505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:33031 (local address 127.15.139.190:42255)
0923 18:00:53.284669 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:53.284673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:53.284691 (+ 18us) server_negotiation.cc:407] Connection header received
0923 18:00:53.284861 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:53.284863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:53.284917 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:53.285101 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:53.285536 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:53.286368 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:53.287166 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:53.287398 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:53.287482 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:53.290683 (+ 3201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:33031: BlockingRecv error: recv got EOF from 127.15.139.130:33031 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 23 18:00:53 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:53 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:53.489827 25105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:53.483637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:56495)
0923 18:00:53.484016 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:53.484030 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:53.484207 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:53.484489 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:53.484492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:53.484702 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:53.484793 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:53.484799 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:53.485636 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:53.485643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:53.486233 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:53.486241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:53.486849 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:53.486852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:53.486943 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:53.487452 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:53.487463 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:53.489708 (+ 2245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":81,"threads_started":1}
W20260923 18:00:53.490088 25103 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:53.483696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:56495 (local address 127.15.139.190:42255)
0923 18:00:53.483866 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:53.483870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:53.484148 (+ 278us) server_negotiation.cc:407] Connection header received
0923 18:00:53.484325 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:53.484328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:53.484384 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:53.484470 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:53.484920 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:53.485509 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:53.486360 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:53.486724 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:53.486845 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:53.489884 (+ 3039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:56495: BlockingRecv error: recv got EOF from 127.15.139.131:56495 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 23 18:00:54 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:54 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:54.096746 25106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:54.090338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:51719)
0923 18:00:54.090699 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:54.090712 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:54.090855 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:54.091167 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:54.091169 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:54.091429 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:54.091661 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:54.091673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:54.092561 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:54.092568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:54.093209 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:54.093218 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:54.093763 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:54.093766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:54.093867 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:54.094465 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:54.094477 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:54.096624 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":81,"threads_started":1}
W20260923 18:00:54.097035 25107 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:54.090434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:51719 (local address 127.15.139.190:42255)
0923 18:00:54.090702 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:54.090707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:54.090818 (+ 111us) server_negotiation.cc:407] Connection header received
0923 18:00:54.090964 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:54.090968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:54.091027 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:54.091133 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:54.091784 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:54.092335 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:54.093346 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:54.093632 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:54.093758 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:54.096794 (+ 3036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:51719: BlockingRecv error: recv got EOF from 127.15.139.129:51719 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":110,"threads_started":1}
Sep 23 18:00:54 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:54 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:54.297544 25108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:54.291747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:39729)
0923 18:00:54.292023 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:54.292038 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:54.292184 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:54.292457 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:54.292461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:54.292701 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:54.292807 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:54.292813 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:54.293588 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:54.293593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:54.294195 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:54.294203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:54.294678 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:54.294681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:54.294764 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:54.295328 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:54.295340 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:54.297372 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":87,"threads_started":1}
W20260923 18:00:54.297637 25107 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:54.291783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:39729 (local address 127.15.139.190:42255)
0923 18:00:54.291947 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:54.291952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:54.292134 (+ 182us) server_negotiation.cc:407] Connection header received
0923 18:00:54.292286 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:54.292289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:54.292340 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:54.292433 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:54.292933 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:54.293472 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:54.294324 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:54.294552 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:54.294677 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:54.297495 (+ 2818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:39729: BlockingRecv error: recv got EOF from 127.15.139.130:39729 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 23 18:00:54 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:54 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 18:00:54.497064 25107 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:54.490807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:42721 (local address 127.15.139.190:42255)
0923 18:00:54.490989 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:54.490994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:54.491159 (+ 165us) server_negotiation.cc:407] Connection header received
0923 18:00:54.491329 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:54.491332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:54.491393 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:54.491526 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:54.491994 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:54.492561 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:54.493404 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:54.493650 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:54.493772 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:54.496955 (+ 3183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:42721: BlockingRecv error: recv got EOF from 127.15.139.131:42721 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260923 18:00:54.497063 25109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:54.490663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:42721)
0923 18:00:54.491062 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:54.491075 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:54.491220 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:54.491553 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:54.491556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:54.491769 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:54.491864 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:54.491870 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:54.492697 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:54.492701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:54.493276 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:54.493284 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:54.493778 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:54.493781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:54.493867 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:54.494436 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:54.494449 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:54.496826 (+ 2377us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":88,"threads_started":1}
Sep 23 18:00:55 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:55 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:55.105024 25110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:55.097793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:32795)
0923 18:00:55.098138 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:55.098152 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:55.098264 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:55.098600 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:55.098603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:55.098895 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:55.099014 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:55.099021 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:55.100205 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:55.100208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:55.100786 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:55.100795 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:55.101570 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:55.101576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:55.101713 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:55.102435 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:55.102448 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:55.104825 (+ 2377us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":105,"threads_started":1}
W20260923 18:00:55.105088 25111 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:55.097899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:32795 (local address 127.15.139.190:42255)
0923 18:00:55.098134 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:55.098138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:55.098240 (+ 102us) server_negotiation.cc:407] Connection header received
0923 18:00:55.098374 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:55.098376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:55.098434 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:55.098519 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:55.099196 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:55.100054 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:55.100981 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:55.101377 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:55.101575 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:55.104940 (+ 3365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:32795: BlockingRecv error: recv got EOF from 127.15.139.129:32795 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
Sep 23 18:00:55 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:55 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:55.304833 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:55.298473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:45129)
0923 18:00:55.298837 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:55.298857 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:55.299063 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:55.299322 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:55.299325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:55.299602 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:55.299712 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:55.299720 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:55.300619 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:55.300622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:55.301221 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:55.301231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:55.301782 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:55.301786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:55.301862 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:55.302418 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:55.302430 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:55.304701 (+ 2271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":105,"threads_started":1}
W20260923 18:00:55.305040 25111 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:55.298502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:45129 (local address 127.15.139.190:42255)
0923 18:00:55.298656 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:55.298659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:55.299006 (+ 347us) server_negotiation.cc:407] Connection header received
0923 18:00:55.299143 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:55.299146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:55.299195 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:55.299274 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:55.299835 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:55.300482 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:55.301355 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:55.301616 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:55.301709 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:55.304888 (+ 3179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:45129: BlockingRecv error: recv got EOF from 127.15.139.130:45129 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 23 18:00:55 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:55 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:55.504827 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:55.498061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:53979)
0923 18:00:55.498616 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:55.498629 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:55.498774 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:55.499180 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:55.499183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:55.499402 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:55.499563 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:55.499574 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:55.500477 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:55.500482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:55.501231 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:55.501241 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:55.501717 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:55.501721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:55.501828 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:55.502361 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:55.502375 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:55.504666 (+ 2291us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":72,"threads_started":1}
W20260923 18:00:55.505034 25111 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:55.498378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:53979 (local address 127.15.139.190:42255)
0923 18:00:55.498522 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:55.498527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:55.498720 (+ 193us) server_negotiation.cc:407] Connection header received
0923 18:00:55.498902 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:55.498905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:55.499025 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:55.499127 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:55.499719 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:55.500324 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:55.501387 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:55.501617 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:55.501733 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:55.504893 (+ 3160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:53979: BlockingRecv error: recv got EOF from 127.15.139.131:53979 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 23 18:00:56 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:56 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:56.112159 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:56.105927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:48729)
0923 18:00:56.106270 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:56.106308 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:56.106437 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:56.106727 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:56.106732 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:56.106968 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:56.107133 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:56.107140 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:56.108078 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:56.108081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:56.108666 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:56.108675 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:56.109126 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:56.109129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:56.109198 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:56.109883 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:56.109897 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:56.111983 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":120,"threads_started":1}
W20260923 18:00:56.112311 25115 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:56.106175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:48729 (local address 127.15.139.190:42255)
0923 18:00:56.106458 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:56.106461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:56.106477 (+ 16us) server_negotiation.cc:407] Connection header received
0923 18:00:56.106533 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:56.106535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:56.106576 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:56.106658 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:56.107262 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:56.107903 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:56.108776 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:56.108990 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:56.109068 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:56.112167 (+ 3099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:48729: BlockingRecv error: recv got EOF from 127.15.139.129:48729 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":124,"threads_started":1}
Sep 23 18:00:56 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:56 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:56.312572 25116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:56.305770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:58497)
0923 18:00:56.306089 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:56.306102 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:56.306241 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:56.306558 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:56.306562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:56.306854 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:56.307038 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:56.307047 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:56.308044 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:56.308048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:56.308624 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:56.308633 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:56.309112 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:56.309115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:56.309211 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:56.309791 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:56.309803 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:56.312378 (+ 2575us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":101,"threads_started":1}
W20260923 18:00:56.312666 25115 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:56.305851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:58497 (local address 127.15.139.190:42255)
0923 18:00:56.306035 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:56.306040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:56.306178 (+ 138us) server_negotiation.cc:407] Connection header received
0923 18:00:56.306345 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:56.306350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:56.306404 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:56.306538 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:56.307172 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:56.307923 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:56.308765 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:56.308992 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:56.309106 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:56.312538 (+ 3432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:58497: BlockingRecv error: recv got EOF from 127.15.139.130:58497 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 23 18:00:56 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:56 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:56.512337 25117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:56.505778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:37715)
0923 18:00:56.506222 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:56.506235 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:56.506383 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:56.506710 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:56.506713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:56.506923 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:56.507027 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:56.507036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:56.508153 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:56.508158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:56.509021 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:56.509031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:56.509455 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:56.509457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:56.509516 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:56.510078 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:56.510151 (+ 73us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:56.512224 (+ 2073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":190,"threads_started":1}
W20260923 18:00:56.512478 25115 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:56.505934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:37715 (local address 127.15.139.190:42255)
0923 18:00:56.506118 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:56.506123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:56.506319 (+ 196us) server_negotiation.cc:407] Connection header received
0923 18:00:56.506499 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:56.506502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:56.506565 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:56.506686 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:56.507157 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:56.507959 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:56.509134 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:56.509354 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:56.509435 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:56.512357 (+ 2922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:37715: BlockingRecv error: recv got EOF from 127.15.139.131:37715 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 23 18:00:57 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:57 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:57.119340 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:57.113220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:43595)
0923 18:00:57.113634 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:57.113653 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:57.113783 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:57.114092 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:57.114096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:57.114317 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:57.114422 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:57.114428 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:57.115257 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:57.115262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:57.115917 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:57.115927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:57.116400 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:57.116403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:57.116487 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:57.117053 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:57.117065 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:57.119171 (+ 2106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":96,"threads_started":1}
W20260923 18:00:57.119446 25119 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:57.113239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:43595 (local address 127.15.139.190:42255)
0923 18:00:57.113616 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:57.113623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:57.113721 (+ 98us) server_negotiation.cc:407] Connection header received
0923 18:00:57.113872 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:57.113877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:57.113937 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:57.114054 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:57.114547 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:57.115121 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:57.116042 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:57.116279 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:57.116394 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:57.119315 (+ 2921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:43595: BlockingRecv error: recv got EOF from 127.15.139.129:43595 (error 108)
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":155,"threads_started":1}
Sep 23 18:00:57 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:57 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:57.319382 25120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:57.313519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:49379)
0923 18:00:57.313812 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:57.313825 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:57.313928 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:57.314196 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:57.314198 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:57.314388 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:57.314510 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:57.314516 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:57.315315 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:57.315318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:57.316206 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:57.316218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:57.316662 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:57.316664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:57.316726 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:57.317260 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:57.317272 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:57.319250 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":125,"threads_started":1}
W20260923 18:00:57.319499 25119 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:57.313616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:49379 (local address 127.15.139.190:42255)
0923 18:00:57.313805 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:57.313810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:57.313904 (+ 94us) server_negotiation.cc:407] Connection header received
0923 18:00:57.314015 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:57.314017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:57.314067 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:57.314157 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:57.314648 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:57.315178 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:57.316325 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:57.316542 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:57.316626 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:57.319383 (+ 2757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:49379: BlockingRecv error: recv got EOF from 127.15.139.130:49379 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 23 18:00:57 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:57 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:57.519014 25121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:57.513205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:53053)
0923 18:00:57.513502 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:57.513516 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:57.513639 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:57.513886 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:57.513888 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:57.514062 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:57.514159 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:57.514164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:57.515056 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:57.515060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:57.515751 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:57.515761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:57.516291 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:57.516294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:57.516385 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:57.516961 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:57.516973 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:57.518905 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":105,"threads_started":1}
W20260923 18:00:57.519173 25119 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:57.513269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:53053 (local address 127.15.139.190:42255)
0923 18:00:57.513445 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:57.513449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:57.513589 (+ 140us) server_negotiation.cc:407] Connection header received
0923 18:00:57.513724 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:57.513726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:57.513773 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:57.513844 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:57.514303 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:57.514918 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:57.515847 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:57.516163 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:57.516270 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:57.519050 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:53053: BlockingRecv error: recv got EOF from 127.15.139.131:53053 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 23 18:00:58 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:58 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:58.131067 25122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:58.120360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:60913)
0923 18:00:58.120653 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:58.120667 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:58.120762 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:58.121005 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:58.121007 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:58.121188 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:58.121287 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:58.121293 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:58.122112 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:58.122115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:58.122692 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:58.122701 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:58.123304 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:58.123307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:58.123374 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:58.128430 (+ 5056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:58.128447 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:58.130877 (+ 2430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":93,"threads_started":1}
W20260923 18:00:58.131218 25123 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:58.120396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:60913 (local address 127.15.139.190:42255)
0923 18:00:58.120644 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:58.120647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:58.120737 (+ 90us) server_negotiation.cc:407] Connection header received
0923 18:00:58.120839 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:58.120842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:58.120884 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:58.120960 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:58.121423 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:58.121979 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:58.122905 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:58.123189 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:58.123282 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:58.131070 (+ 7788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:60913: BlockingRecv error: recv got EOF from 127.15.139.129:60913 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
Sep 23 18:00:58 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:58 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:58.327430 25124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:58.320401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:48095)
0923 18:00:58.320837 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:58.320871 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:58.321074 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:58.321421 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:58.321424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:58.321656 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:58.321806 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:58.321817 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:58.322945 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:58.322948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:58.323722 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:58.323731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:58.324232 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:58.324235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:58.324368 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:58.325102 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:58.325114 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:58.327254 (+ 2140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":87,"threads_started":1}
W20260923 18:00:58.327488 25123 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:58.320543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:48095 (local address 127.15.139.190:42255)
0923 18:00:58.320757 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:58.320762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:58.321040 (+ 278us) server_negotiation.cc:407] Connection header received
0923 18:00:58.321173 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:58.321177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:58.321245 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:58.321349 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:58.321960 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:58.322832 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:58.323826 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:58.324119 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:58.324212 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:58.327380 (+ 3168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:48095: BlockingRecv error: recv got EOF from 127.15.139.130:48095 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Sep 23 18:00:58 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:58 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:58.525887 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:58.519938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:47541)
0923 18:00:58.520276 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:58.520294 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:58.520398 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:58.520699 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:58.520701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:58.520939 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:58.521035 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:58.521041 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:58.521795 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:58.521802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:58.522369 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:58.522377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:58.522853 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:58.522856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:58.522948 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:58.523490 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:58.523547 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:58.525720 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":81,"threads_started":1}
W20260923 18:00:58.525939 25123 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:58.520100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:47541 (local address 127.15.139.190:42255)
0923 18:00:58.520253 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:58.520257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:58.520376 (+ 119us) server_negotiation.cc:407] Connection header received
0923 18:00:58.520505 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:58.520515 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:58.520561 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:58.520676 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:58.521155 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:58.521687 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:58.522488 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:58.522720 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:58.522832 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:58.525833 (+ 3001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:47541: BlockingRecv error: recv got EOF from 127.15.139.131:47541 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 23 18:00:59 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:59 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:59.138170 25126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:59.132057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:40597)
0923 18:00:59.132365 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:59.132380 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:59.132486 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:59.132879 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:59.132883 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:59.133159 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:59.133258 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:59.133264 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:59.134316 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:59.134319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:59.134902 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:59.134910 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:59.135326 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:59.135329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:59.135394 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:59.136002 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:59.136016 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:59.137987 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":92,"threads_started":1}
W20260923 18:00:59.138233 25127 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:59.132210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:40597 (local address 127.15.139.190:42255)
0923 18:00:59.132433 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:59.132437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:59.132452 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:00:59.132616 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:59.132619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:59.132675 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:59.132783 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:59.133401 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:59.134200 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:59.135003 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:59.135221 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:59.135297 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:59.138115 (+ 2818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:40597: BlockingRecv error: recv got EOF from 127.15.139.129:40597 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":59,"threads_started":1}
Sep 23 18:00:59 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:59 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:59.333668 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:59.328287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:53119)
0923 18:00:59.328566 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:59.328580 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:59.328682 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:59.328912 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:59.328914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:59.329109 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:59.329208 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:59.329215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:59.329917 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:59.329919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:59.330469 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:59.330476 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:59.331040 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:59.331044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:59.331144 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:59.331743 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:59.331758 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:59.333555 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":94,"threads_started":1}
W20260923 18:00:59.333843 25127 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:59.328333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:53119 (local address 127.15.139.190:42255)
0923 18:00:59.328474 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:59.328478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:59.328649 (+ 171us) server_negotiation.cc:407] Connection header received
0923 18:00:59.328759 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:59.328761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:59.328804 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:59.328893 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:59.329312 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:59.329826 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:59.330599 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:59.330908 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:59.331022 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:59.333690 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:53119: BlockingRecv error: recv got EOF from 127.15.139.130:53119 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 23 18:00:59 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:00:59 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:00:59.532706 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:00:59.526715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:39505)
0923 18:00:59.527072 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0923 18:00:59.527087 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:00:59.527194 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:00:59.527436 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:00:59.527439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:00:59.527714 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:00:59.527815 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:59.527821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:59.528613 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:59.528615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:59.529214 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:00:59.529222 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:00:59.529647 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:00:59.529650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:00:59.529727 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:59.530370 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:00:59.530385 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:00:59.532569 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":80,"threads_started":1}
W20260923 18:00:59.532841 25127 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:00:59.526800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:39505 (local address 127.15.139.190:42255)
0923 18:00:59.527039 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0923 18:00:59.527042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:00:59.527161 (+ 119us) server_negotiation.cc:407] Connection header received
0923 18:00:59.527274 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:00:59.527276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:00:59.527323 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:00:59.527393 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:00:59.527931 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:59.528507 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:59.529320 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:00:59.529536 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:00:59.529615 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:00:59.532720 (+ 3105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:39505: BlockingRecv error: recv got EOF from 127.15.139.131:39505 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 23 18:01:00 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:01:00 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:01:00.145406 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:00.139197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:41499)
0923 18:01:00.139614 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:00.139630 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:00.139748 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:00.140118 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:00.140121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:00.140307 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:00.140410 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:00.140416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:00.141213 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:00.141215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:00.141829 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:00.141843 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:00.142400 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:00.142404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:00.142471 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:00.143027 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:00.143039 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:00.145277 (+ 2238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":105,"threads_started":1}
W20260923 18:01:00.145632 25131 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:00.139275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:41499 (local address 127.15.139.190:42255)
0923 18:01:00.139621 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:00.139625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:00.139717 (+ 92us) server_negotiation.cc:407] Connection header received
0923 18:01:00.139891 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:00.139894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:00.139945 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:00.140035 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:00.140543 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:00.141108 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:00.142048 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:00.142292 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:00.142383 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:00.145427 (+ 3044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:41499: BlockingRecv error: recv got EOF from 127.15.139.129:41499 (error 108)
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":71,"threads_started":1}
Sep 23 18:01:00 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:01:00 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:01:00.340936 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:00.334544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:57555)
0923 18:01:00.334960 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:00.334978 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:00.335144 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:00.335573 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:00.335577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:00.335841 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:00.335992 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:00.336005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:00.337025 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:00.337028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:00.337636 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:00.337645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:00.338087 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:00.338089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:00.338178 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:00.338754 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:00.338768 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:00.340763 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":103,"threads_started":1}
W20260923 18:01:00.341001 25131 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:00.334730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:57555 (local address 127.15.139.190:42255)
0923 18:01:00.334995 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:00.335012 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:00.335110 (+ 98us) server_negotiation.cc:407] Connection header received
0923 18:01:00.335270 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:00.335273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:00.335353 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:00.335442 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:00.336138 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:00.336922 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:00.337780 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:00.337987 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:00.338078 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:00.340895 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:57555: BlockingRecv error: recv got EOF from 127.15.139.130:57555 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Sep 23 18:01:00 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:01:00 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:01:00.540102 25133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:00.533641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:45815)
0923 18:01:00.534013 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:00.534034 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:00.534165 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:00.534471 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:00.534474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:00.534740 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:00.534843 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:00.534850 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:00.535792 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:00.535798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:00.536621 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:00.536631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:00.537142 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:00.537145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:00.537270 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:00.537783 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:00.537795 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:00.539969 (+ 2174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":93,"threads_started":1}
W20260923 18:01:00.540234 25131 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:00.533809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:45815 (local address 127.15.139.190:42255)
0923 18:01:00.534073 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:00.534080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:00.534102 (+ 22us) server_negotiation.cc:407] Connection header received
0923 18:01:00.534291 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:00.534294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:00.534352 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:00.534443 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:00.534970 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:00.535666 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:00.536743 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:00.537016 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:00.537123 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:00.540111 (+ 2988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:45815: BlockingRecv error: recv got EOF from 127.15.139.131:45815 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 23 18:01:01 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:01:01 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:01:01.152940 25134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:01.146287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:41749)
0923 18:01:01.146579 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:01.146593 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:01.146779 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:01.147126 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:01.147129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:01.147382 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:01.147558 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:01.147567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:01.148524 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:01.148529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:01.149134 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:01.149143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:01.149899 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:01.149903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:01.150024 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:01.150710 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:01.150724 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:01.152825 (+ 2101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":84,"threads_started":1}
W20260923 18:01:01.153199 25135 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:01.146362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:41749 (local address 127.15.139.190:42255)
0923 18:01:01.146633 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:01.146636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:01.146731 (+ 95us) server_negotiation.cc:407] Connection header received
0923 18:01:01.146871 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:01.146873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:01.146946 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:01.147047 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:01.147693 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:01.148417 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:01.149265 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:01.149513 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:01.149622 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:01.153055 (+ 3433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:41749: BlockingRecv error: recv got EOF from 127.15.139.129:41749 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":72,"threads_started":1}
Sep 23 18:01:01 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:01:01 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:01:01.348038 25136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:01.341766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:36849)
0923 18:01:01.342159 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:01.342180 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:01.342323 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:01.342647 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:01.342651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:01.342884 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:01.342981 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:01.342988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:01.343893 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:01.343899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:01.344547 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:01.344556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:01.345087 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:01.345091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:01.345194 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:01.345742 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:01.345754 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:01.347885 (+ 2131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":123,"threads_started":1}
W20260923 18:01:01.348232 25135 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:01.341969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:36849 (local address 127.15.139.190:42255)
0923 18:01:01.342187 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:01.342195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:01.342253 (+ 58us) server_negotiation.cc:407] Connection header received
0923 18:01:01.342466 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:01.342469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:01.342519 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:01.342620 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:01.343127 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:01.343777 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:01.344677 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:01.344927 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:01.345072 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:01.348060 (+ 2988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:36849: BlockingRecv error: recv got EOF from 127.15.139.130:36849 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Sep 23 18:01:01 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:01:01 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:01:01.547065 25137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:01.540977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:40411)
0923 18:01:01.541341 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:01.541354 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:01.541481 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:01.541775 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:01.541778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:01.542005 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:01.542100 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:01.542106 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:01.542894 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:01.542899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:01.543474 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:01.543482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:01.543978 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:01.543981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:01.544113 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:01.544601 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:01.544612 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:01.546881 (+ 2269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":96,"threads_started":1}
W20260923 18:01:01.547286 25135 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:01.541109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:40411 (local address 127.15.139.190:42255)
0923 18:01:01.541246 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:01.541250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:01.541429 (+ 179us) server_negotiation.cc:407] Connection header received
0923 18:01:01.541611 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:01.541613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:01.541659 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:01.541753 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:01.542221 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:01.542779 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:01.543628 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:01.543854 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:01.543962 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:01.547112 (+ 3150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:40411: BlockingRecv error: recv got EOF from 127.15.139.131:40411 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 23 18:01:02 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:01:02 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:01:02.159220 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:02.153722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:35303)
0923 18:01:02.154009 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:02.154023 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:02.154150 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:02.154417 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:02.154419 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:02.154600 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:02.154695 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:02.154700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:02.155430 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:02.155432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:02.156046 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:02.156056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:02.156585 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:02.156588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:02.156657 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:02.157223 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:02.157234 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:02.159093 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":98,"threads_started":1}
W20260923 18:01:02.159541 25139 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:02.153787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:35303 (local address 127.15.139.190:42255)
0923 18:01:02.154104 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:02.154109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:02.154131 (+ 22us) server_negotiation.cc:407] Connection header received
0923 18:01:02.154246 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:02.154248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:02.154297 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:02.154383 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:02.154800 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:02.155331 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:02.156208 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:02.156470 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:02.156562 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:02.159331 (+ 2769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:35303: BlockingRecv error: recv got EOF from 127.15.139.129:35303 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":63,"threads_started":1}
Sep 23 18:01:02 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:01:02 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:01:02.354997 25140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:02.348916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:44293)
0923 18:01:02.349310 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:02.349324 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:02.349468 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:02.349800 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:02.349803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:02.350031 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:02.350128 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:02.350134 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:02.350897 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:02.350900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:02.351650 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:02.351661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:02.352142 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:02.352145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:02.352214 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:02.352780 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:02.352792 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:02.354844 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":122,"threads_started":1}
W20260923 18:01:02.355144 25139 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:02.349035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:44293 (local address 127.15.139.190:42255)
0923 18:01:02.349176 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:02.349180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:02.349408 (+ 228us) server_negotiation.cc:407] Connection header received
0923 18:01:02.349560 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:02.349562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:02.349609 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:02.349685 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:02.350245 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:02.350767 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:02.351793 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:02.352023 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:02.352118 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:02.354983 (+ 2865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:44293: BlockingRecv error: recv got EOF from 127.15.139.130:44293 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 23 18:01:02 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:01:02 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.131@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:01:02.553581 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:02.547949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.131:52741)
0923 18:01:02.548241 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:02.548255 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:02.548357 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:02.548638 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:02.548640 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:02.548814 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:02.548910 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:02.548915 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:02.549728 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:02.549730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:02.550290 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:02.550299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:02.550775 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:02.550777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:02.550839 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:02.551369 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:02.551381 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:02.553442 (+ 2061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":115,"threads_started":1}
W20260923 18:01:02.553750 25139 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:02.547984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:52741 (local address 127.15.139.190:42255)
0923 18:01:02.548137 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:02.548143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:02.548333 (+ 190us) server_negotiation.cc:407] Connection header received
0923 18:01:02.548438 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:02.548440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:02.548517 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:02.548586 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:02.549023 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:02.549615 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:02.550402 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:02.550673 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:02.550763 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:02.553618 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.131:52741: BlockingRecv error: recv got EOF from 127.15.139.131:52741 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 23 18:01:03 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:01:03 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.129@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:01:03.175992 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:03.160321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.129:41965)
0923 18:01:03.160614 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:03.160628 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:03.160732 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:03.161156 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:03.161159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:03.161369 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:03.161466 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:03.161471 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.162471 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.162477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:03.163130 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:03.163140 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.163773 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.163776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:03.163840 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:03.173495 (+ 9655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:03.173513 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:03.175861 (+ 2348us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":104,"threads_started":1}
W20260923 18:01:03.176219 25143 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:03.160392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:41965 (local address 127.15.139.190:42255)
0923 18:01:03.160661 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:03.160665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:03.160683 (+ 18us) server_negotiation.cc:407] Connection header received
0923 18:01:03.160852 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:03.160855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:03.160915 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:03.161045 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:03.161566 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.162326 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.163250 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.163656 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.163749 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:03.176007 (+ 12258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.129:41965: BlockingRecv error: recv got EOF from 127.15.139.129:41965 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":119,"threads_started":1}
Sep 23 18:01:03 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 18:01:03 dist-test-slave-vr57 krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.139.130@KRBTEST.COM for oryx/127.15.139.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 18:01:03.361793 25144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:03.356059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42255 (local address 127.15.139.130:52245)
0923 18:01:03.356319 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:03.356334 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:03.356436 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:03.356702 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:03.356705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:03.356886 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:03.356984 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:03.356990 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.357852 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.357855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:03.358409 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:03.358417 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.358965 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.358968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:03.359034 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:03.359643 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:03.359661 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:03.361608 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.139.190:42255: Server oryx/127.15.139.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":98,"threads_started":1}
W20260923 18:01:03.361853 25143 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:03.356209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:52245 (local address 127.15.139.190:42255)
0923 18:01:03.356352 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:03.356356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:03.356394 (+ 38us) server_negotiation.cc:407] Connection header received
0923 18:01:03.356522 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:03.356524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:03.356579 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:03.356657 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:03.357100 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.357746 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.358585 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.358860 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.358947 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:03.361736 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.139.130:52245: BlockingRecv error: recv got EOF from 127.15.139.130:52245 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:01:03.445928 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 24668
I20260923 18:01:03.452507 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 24820
I20260923 18:01:03.458086 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 24952
I20260923 18:01:03.463223 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 24598
2026-09-23T18:01:03Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18154 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-23T18:01:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:01:03Z Disabled control of system clock
I20260923 18:01:03.495255 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:35923
--webserver_interface=127.15.139.190
--webserver_port=0
--builtin_ntp_servers=127.15.139.148:39299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:35923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 18:01:03.612607 25148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:03.612872 25148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:03.612908 25148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:03.616950 25148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:01:03.617025 25148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:03.617040 25148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:03.617057 25148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:01:03.617074 25148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:01:03.622035 25148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:39299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:35923
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:35923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:03.623788 25148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:03.625427 25148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:03.635104 25153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:03.635141 25154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:03.635167 25156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:03.635299 25148 server_base.cc:1034] running on GCE node
I20260923 18:01:03.635933 25148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:03.637115 25148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:03.638311 25148 hybrid_clock.cc:648] HybridClock initialized: now 1790186463638292 us; error 36 us; skew 500 ppm
I20260923 18:01:03.640413 25148 webserver.cc:460] Webserver started at http://127.15.139.190:45893/ using document root <none> and password file <none>
I20260923 18:01:03.641089 25148 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:03.641178 25148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:03.641409 25148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:01:03.643468 25148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "c0745b66a7e04eb9850a79ed5181e71d"
format_stamp: "Formatted at 2026-09-23 18:01:03 on dist-test-slave-vr57"
server_key: "e41b5423bf68d07f81496cc18574fd5e"
server_key_iv: "a91e1b181cb8b785234b8a1bcfa4395c"
server_key_version: "encryptionkey@0"
I20260923 18:01:03.644093 25148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "c0745b66a7e04eb9850a79ed5181e71d"
format_stamp: "Formatted at 2026-09-23 18:01:03 on dist-test-slave-vr57"
server_key: "e41b5423bf68d07f81496cc18574fd5e"
server_key_iv: "a91e1b181cb8b785234b8a1bcfa4395c"
server_key_version: "encryptionkey@0"
I20260923 18:01:03.647863 25148 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260923 18:01:03.650590 25162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:03.651961 25148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:01:03.652132 25148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "c0745b66a7e04eb9850a79ed5181e71d"
format_stamp: "Formatted at 2026-09-23 18:01:03 on dist-test-slave-vr57"
server_key: "e41b5423bf68d07f81496cc18574fd5e"
server_key_iv: "a91e1b181cb8b785234b8a1bcfa4395c"
server_key_version: "encryptionkey@0"
I20260923 18:01:03.652261 25148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:03.663949 25148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:03.664624 25148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:03.664816 25148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:03.672156 25148 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:35923
I20260923 18:01:03.672183 25214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:35923 every 8 connection(s)
I20260923 18:01:03.673110 25148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:01:03.676544 25215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:01:03.681550 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 25148
I20260923 18:01:03.681715 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:01:03.681999 15918 external_mini_cluster.cc:1428] Setting key ce317e099542fa55ab6346ebaf5ed774
I20260923 18:01:03.682444 25215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d: Bootstrap starting.
I20260923 18:01:03.685179 25215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d: Neither blocks nor log segments found. Creating new log.
I20260923 18:01:03.685950 25215 log.cc:824] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d: Log is configured to *not* fsync() on all Append() calls
I20260923 18:01:03.688025 25215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d: No bootstrap required, opened a new log
I20260923 18:01:03.691258 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:03.683486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59880 (local address 127.15.139.190:35923)
0923 18:01:03.684226 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:03.684240 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:03.684275 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:01:03.684886 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:03.684902 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:03.685358 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:03.685690 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:03.686166 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.687129 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.688117 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.688595 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.688760 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:03.689606 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:03.689617 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:03.689630 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:01:03.689997 (+ 367us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:03.690048 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:01:03.690067 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:03.690183 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:03.690296 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 18:01:03.690542 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":147,"threads_started":1}
I20260923 18:01:03.691948 25215 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0745b66a7e04eb9850a79ed5181e71d" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35923 } }
I20260923 18:01:03.692201 25215 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:01:03.692281 25215 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0745b66a7e04eb9850a79ed5181e71d, State: Initialized, Role: FOLLOWER
I20260923 18:01:03.692744 25215 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0745b66a7e04eb9850a79ed5181e71d" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35923 } }
I20260923 18:01:03.692914 25215 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:01:03.692996 25215 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:01:03.693122 25215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:01:03.694193 25215 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0745b66a7e04eb9850a79ed5181e71d" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35923 } }
I20260923 18:01:03.694545 25215 leader_election.cc:302] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0745b66a7e04eb9850a79ed5181e71d; no voters:
I20260923 18:01:03.694868 25215 leader_election.cc:288] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:01:03.694973 25220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:01:03.695232 25220 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [term 1 LEADER]: Becoming Leader. State: Replica: c0745b66a7e04eb9850a79ed5181e71d, State: Running, Role: LEADER
I20260923 18:01:03.695631 25220 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0745b66a7e04eb9850a79ed5181e71d" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35923 } }
I20260923 18:01:03.696230 25215 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:01:03.697403 25222 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0745b66a7e04eb9850a79ed5181e71d. Latest consensus state: current_term: 1 leader_uuid: "c0745b66a7e04eb9850a79ed5181e71d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0745b66a7e04eb9850a79ed5181e71d" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35923 } } }
I20260923 18:01:03.697417 25221 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0745b66a7e04eb9850a79ed5181e71d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0745b66a7e04eb9850a79ed5181e71d" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35923 } } }
I20260923 18:01:03.697640 25221 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [sys.catalog]: This master's current role is: LEADER
I20260923 18:01:03.697956 25222 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d [sys.catalog]: This master's current role is: LEADER
I20260923 18:01:03.698083 25229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:01:03.701433 25229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:01:03.707296 25229 catalog_manager.cc:1347] Generated new cluster ID: 1835416349ce4bf1b715f09ef1761423
I20260923 18:01:03.707376 25229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:01:03.720340 25229 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:01:03.721314 25229 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:01:03.726925 25229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0745b66a7e04eb9850a79ed5181e71d: Generated new TSK 0
I20260923 18:01:03.727773 25229 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:01:03.740053 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35923
--builtin_ntp_servers=127.15.139.148:39299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 18:01:03.871925 25239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:03.872177 25239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:03.872215 25239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:03.876298 25239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:03.876353 25239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:03.876441 25239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:01:03.881114 25239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:39299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:03.882138 25239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:03.883462 25239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:03.893568 25247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:03.893568 25245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:03.893607 25244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:03.893947 25239 server_base.cc:1034] running on GCE node
I20260923 18:01:03.894333 25239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:03.894934 25239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:03.896135 25239 hybrid_clock.cc:648] HybridClock initialized: now 1790186463896103 us; error 55 us; skew 500 ppm
I20260923 18:01:03.898236 25239 webserver.cc:460] Webserver started at http://127.15.139.129:39095/ using document root <none> and password file <none>
I20260923 18:01:03.898815 25239 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:03.898905 25239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:03.899133 25239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:01:03.901234 25239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "89dd0bde56204032a91d734c0207b0a3"
format_stamp: "Formatted at 2026-09-23 18:01:03 on dist-test-slave-vr57"
server_key: "d7002968276979086ef98d536b59316f"
server_key_iv: "8aea029bdf2988559334a4013ad3bddc"
server_key_version: "encryptionkey@0"
I20260923 18:01:03.901796 25239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "89dd0bde56204032a91d734c0207b0a3"
format_stamp: "Formatted at 2026-09-23 18:01:03 on dist-test-slave-vr57"
server_key: "d7002968276979086ef98d536b59316f"
server_key_iv: "8aea029bdf2988559334a4013ad3bddc"
server_key_version: "encryptionkey@0"
I20260923 18:01:03.905505 25239 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260923 18:01:03.908149 25253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:03.909418 25239 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:01:03.909562 25239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "89dd0bde56204032a91d734c0207b0a3"
format_stamp: "Formatted at 2026-09-23 18:01:03 on dist-test-slave-vr57"
server_key: "d7002968276979086ef98d536b59316f"
server_key_iv: "8aea029bdf2988559334a4013ad3bddc"
server_key_version: "encryptionkey@0"
I20260923 18:01:03.909679 25239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:03.932092 25239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:03.932883 25239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:03.933117 25239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:03.933745 25239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:01:03.934725 25239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:01:03.934798 25239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:03.934872 25239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:01:03.934926 25239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:03.943976 25239 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:46221
I20260923 18:01:03.943994 25366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:46221 every 8 connection(s)
I20260923 18:01:03.944990 25239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:01:03.947743 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 25239
I20260923 18:01:03.947885 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:01:03.948213 15918 external_mini_cluster.cc:1428] Setting key fd2a03420d43532244d3a77941731b45
I20260923 18:01:03.950228 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35923
--builtin_ntp_servers=127.15.139.148:39299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260923 18:01:03.955056 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:03.946947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:39929 (local address 127.15.139.190:35923)
0923 18:01:03.947115 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:03.947120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:03.948336 (+ 1216us) server_negotiation.cc:407] Connection header received
0923 18:01:03.950003 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:03.950006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:03.950059 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:03.950147 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:03.951188 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.951840 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.953025 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.953261 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.953373 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:03.954562 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:03.954570 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:03.954572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:01:03.954584 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:03.954604 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:01:03.954614 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:03.954695 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:03.954854 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 18:01:03.954913 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260923 18:01:03.956048 25369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:03.947375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:35923 (local address 127.15.139.129:39929)
0923 18:01:03.948177 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:03.948230 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:03.949788 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:03.950372 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:03.950381 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:03.950498 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:03.951030 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:03.951043 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.951981 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.951984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:03.952894 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:03.952903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:03.953417 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:03.953421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:03.953583 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:03.954308 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:01:03.954309 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:03.954348 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:01:03.954410 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:01:03.954709 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:01:03.954721 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:01:03.954731 (+ 10us) client_negotiation.cc:769] Sending connection context
0923 18:01:03.954854 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 18:01:03.955138 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":196,"threads_started":1}
I20260923 18:01:03.957329 25367 heartbeater.cc:347] Connected to a master server at 127.15.139.190:35923
I20260923 18:01:03.957576 25367 heartbeater.cc:464] Registering TS with master...
I20260923 18:01:03.958092 25367 heartbeater.cc:511] Master 127.15.139.190:35923 requested a full tablet report, sending...
I20260923 18:01:03.959806 25179 ts_manager.cc:194] Registered new tserver with Master: 89dd0bde56204032a91d734c0207b0a3 (127.15.139.129:46221)
I20260923 18:01:03.961556 25179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.139.129:39929
W20260923 18:01:04.070147 25370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:04.070408 25370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:04.070446 25370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:04.074431 25370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:04.074481 25370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:04.074532 25370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:01:04.079330 25370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:39299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:04.080430 25370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:04.081738 25370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:04.092401 25378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:04.092399 25375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:04.092397 25376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:04.092759 25370 server_base.cc:1034] running on GCE node
I20260923 18:01:04.093252 25370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:04.093932 25370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:04.095129 25370 hybrid_clock.cc:648] HybridClock initialized: now 1790186464095105 us; error 36 us; skew 500 ppm
I20260923 18:01:04.097141 25370 webserver.cc:460] Webserver started at http://127.15.139.130:40449/ using document root <none> and password file <none>
I20260923 18:01:04.097844 25370 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:04.097915 25370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:04.098088 25370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:01:04.100339 25370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "f1c8a37f87484f87aa09cff940ebefed"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "8d14a5fa23c50a00c90c19a688db8ff3"
server_key_iv: "c98570c721f201c15289cd8ebb02ba6a"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.100888 25370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "f1c8a37f87484f87aa09cff940ebefed"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "8d14a5fa23c50a00c90c19a688db8ff3"
server_key_iv: "c98570c721f201c15289cd8ebb02ba6a"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.104621 25370 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260923 18:01:04.107177 25384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.108444 25370 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:01:04.108574 25370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "f1c8a37f87484f87aa09cff940ebefed"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "8d14a5fa23c50a00c90c19a688db8ff3"
server_key_iv: "c98570c721f201c15289cd8ebb02ba6a"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.108681 25370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:04.130633 25370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:04.131357 25370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:04.131613 25370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:04.132237 25370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:01:04.133210 25370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:01:04.133282 25370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.133349 25370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:01:04.133392 25370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.142838 25370 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:40767
I20260923 18:01:04.142858 25497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:40767 every 8 connection(s)
I20260923 18:01:04.143889 25370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:01:04.146981 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 25370
I20260923 18:01:04.147094 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:01:04.147432 15918 external_mini_cluster.cc:1428] Setting key a73e8fd009ef202ae326338ca2f1a5d9
I20260923 18:01:04.149519 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35923
--builtin_ntp_servers=127.15.139.148:39299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260923 18:01:04.154316 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:04.145831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:49561 (local address 127.15.139.190:35923)
0923 18:01:04.145994 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:04.145999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:04.147463 (+ 1464us) server_negotiation.cc:407] Connection header received
0923 18:01:04.148969 (+ 1506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:04.148972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:04.149024 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:04.149141 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:04.150122 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:04.150732 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:04.152006 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:04.152352 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:04.152472 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:04.153700 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:04.153705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:04.153712 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:01:04.153729 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:04.153756 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:01:04.153768 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:04.153860 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:04.154011 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 18:01:04.154089 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 18:01:04.155143 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:04.146448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:35923 (local address 127.15.139.130:49561)
0923 18:01:04.147270 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:04.147333 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:04.148749 (+ 1416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:04.149291 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:04.149298 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:04.149386 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:04.149972 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:04.149986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:04.150897 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:04.150900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:04.151882 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:04.151891 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:04.152531 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:04.152535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:04.152692 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:04.153372 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:01:04.153374 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:04.153415 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:01:04.153514 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:01:04.153866 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:01:04.153873 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:01:04.153878 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 18:01:04.153978 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 18:01:04.154264 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":122,"threads_started":1}
I20260923 18:01:04.156369 25498 heartbeater.cc:347] Connected to a master server at 127.15.139.190:35923
I20260923 18:01:04.156652 25498 heartbeater.cc:464] Registering TS with master...
I20260923 18:01:04.157388 25498 heartbeater.cc:511] Master 127.15.139.190:35923 requested a full tablet report, sending...
I20260923 18:01:04.158617 25179 ts_manager.cc:194] Registered new tserver with Master: f1c8a37f87484f87aa09cff940ebefed (127.15.139.130:40767)
I20260923 18:01:04.159320 25179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.139.130:49561
W20260923 18:01:04.263607 25501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:04.263866 25501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:04.263903 25501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:04.267889 25501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:04.267939 25501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:04.268029 25501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:01:04.272753 25501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:39299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:04.273820 25501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:04.275209 25501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:04.284793 25509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:04.284793 25507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:04.284795 25506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:04.285224 25501 server_base.cc:1034] running on GCE node
I20260923 18:01:04.285634 25501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:04.286372 25501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:04.287669 25501 hybrid_clock.cc:648] HybridClock initialized: now 1790186464287624 us; error 54 us; skew 500 ppm
I20260923 18:01:04.289809 25501 webserver.cc:460] Webserver started at http://127.15.139.131:36319/ using document root <none> and password file <none>
I20260923 18:01:04.290544 25501 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:04.290638 25501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:04.290913 25501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:01:04.293078 25501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "aa31122ea99b4bca90acd2209d73dda1"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "12de032519e9a9e8bfd5ec62463cb808"
server_key_iv: "df3bb5ea3b3698e44df799bdb6a03ba9"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.293653 25501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "aa31122ea99b4bca90acd2209d73dda1"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "12de032519e9a9e8bfd5ec62463cb808"
server_key_iv: "df3bb5ea3b3698e44df799bdb6a03ba9"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.297739 25501 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:01:04.300390 25515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.301645 25501 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:01:04.301767 25501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "aa31122ea99b4bca90acd2209d73dda1"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "12de032519e9a9e8bfd5ec62463cb808"
server_key_iv: "df3bb5ea3b3698e44df799bdb6a03ba9"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.301879 25501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:04.320264 25501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:04.320989 25501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:04.321201 25501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:04.321825 25501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:01:04.322778 25501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:01:04.322825 25501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.322892 25501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:01:04.322924 25501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.331887 25501 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:45733
I20260923 18:01:04.331923 25628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:45733 every 8 connection(s)
I20260923 18:01:04.332798 25501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:01:04.336678 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 25501
I20260923 18:01:04.336797 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:01:04.337148 15918 external_mini_cluster.cc:1428] Setting key 38f4290f33c383c295ffc6486c169222
I20260923 18:01:04.342497 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:04.334624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:44331 (local address 127.15.139.190:35923)
0923 18:01:04.334763 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:04.334767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:04.335847 (+ 1080us) server_negotiation.cc:407] Connection header received
0923 18:01:04.337586 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:04.337590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:04.337650 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:04.337738 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:04.338762 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:04.339299 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:04.340633 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:04.340856 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:04.340940 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:04.342143 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:04.342145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:04.342147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:01:04.342159 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:04.342174 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:01:04.342181 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:04.342231 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:04.342339 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 18:01:04.342388 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 18:01:04.343395 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:04.335019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:35923 (local address 127.15.139.131:44331)
0923 18:01:04.335693 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:04.335735 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:04.337339 (+ 1604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:04.337926 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:04.337933 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:04.338020 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:04.338625 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:04.338639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:04.339420 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:04.339423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:04.340515 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:04.340525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:04.341060 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:04.341063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:04.341210 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:04.341869 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:01:04.341871 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:04.341911 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:01:04.341975 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:01:04.342232 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:01:04.342236 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:01:04.342240 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 18:01:04.342326 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 18:01:04.342578 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":109,"threads_started":1}
I20260923 18:01:04.344638 25629 heartbeater.cc:347] Connected to a master server at 127.15.139.190:35923
I20260923 18:01:04.344882 25629 heartbeater.cc:464] Registering TS with master...
I20260923 18:01:04.345392 25629 heartbeater.cc:511] Master 127.15.139.190:35923 requested a full tablet report, sending...
I20260923 18:01:04.346441 25179 ts_manager.cc:194] Registered new tserver with Master: aa31122ea99b4bca90acd2209d73dda1 (127.15.139.131:45733)
I20260923 18:01:04.347087 25179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.139.131:44331
I20260923 18:01:04.351267 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 18:01:04.354084 25218 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:04.353193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59884 (local address 127.15.139.190:35923)
0923 18:01:04.353351 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:04.353360 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:04.353458 (+ 98us) server_negotiation.cc:407] Connection header received
0923 18:01:04.353543 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:04.353545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:04.353594 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:04.353679 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:04.353927 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59884: BlockingRecv error: recv got EOF from 127.0.0.1:59884 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 18:01:04.354190 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 25239
I20260923 18:01:04.360227 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 25370
I20260923 18:01:04.365527 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 25501
I20260923 18:01:04.370769 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 25148
2026-09-23T18:01:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (905 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-23T18:01:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:01:04Z Disabled control of system clock
I20260923 18:01:04.400342 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:44599
--webserver_interface=127.15.139.190
--webserver_port=0
--builtin_ntp_servers=127.15.139.148:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:44599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 18:01:04.511330 25640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:04.511677 25640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:04.511719 25640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:04.515817 25640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:01:04.515872 25640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:04.515888 25640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:04.515904 25640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:01:04.515920 25640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:01:04.521009 25640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:44599
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:44599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:04.522045 25640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:04.523370 25640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:04.531901 25646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:04.531916 25645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:04.531901 25648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:04.532015 25640 server_base.cc:1034] running on GCE node
I20260923 18:01:04.532792 25640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:04.533809 25640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:04.534966 25640 hybrid_clock.cc:648] HybridClock initialized: now 1790186464534956 us; error 30 us; skew 500 ppm
I20260923 18:01:04.536880 25640 webserver.cc:460] Webserver started at http://127.15.139.190:42079/ using document root <none> and password file <none>
I20260923 18:01:04.537446 25640 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:04.537506 25640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:04.537750 25640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:01:04.540072 25640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "7401be5ddd2a49fda0cf26d121e0433e"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "90da46351cd10368c9a52a1969476f9a"
server_key_iv: "e24302cd5db3f88514f14559c578690f"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.540643 25640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "7401be5ddd2a49fda0cf26d121e0433e"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "90da46351cd10368c9a52a1969476f9a"
server_key_iv: "e24302cd5db3f88514f14559c578690f"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.544312 25640 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260923 18:01:04.546854 25654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.548319 25640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:01:04.548496 25640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "7401be5ddd2a49fda0cf26d121e0433e"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "90da46351cd10368c9a52a1969476f9a"
server_key_iv: "e24302cd5db3f88514f14559c578690f"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.548617 25640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:04.553359 25640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:04.553968 25640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:04.554153 25640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:04.561185 25640 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:44599
I20260923 18:01:04.561203 25706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:44599 every 8 connection(s)
I20260923 18:01:04.562103 25640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:01:04.565294 25707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:01:04.566044 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 25640
I20260923 18:01:04.566191 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:01:04.566516 15918 external_mini_cluster.cc:1428] Setting key baf06c1f36fb2942e38f0033436d45b0
I20260923 18:01:04.571604 25707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e: Bootstrap starting.
I20260923 18:01:04.572192 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:04.568047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.15.139.190:44599)
0923 18:01:04.568686 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:04.568701 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:04.568735 (+ 34us) server_negotiation.cc:407] Connection header received
0923 18:01:04.569354 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:04.569370 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:04.569709 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:04.570043 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:04.570835 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:04.570839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:04.570851 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:01:04.570862 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:04.570917 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:01:04.570933 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:04.571083 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:04.571163 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 18:01:04.571394 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":122,"threads_started":1}
I20260923 18:01:04.574011 25707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e: Neither blocks nor log segments found. Creating new log.
I20260923 18:01:04.574813 25707 log.cc:824] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e: Log is configured to *not* fsync() on all Append() calls
I20260923 18:01:04.576645 25707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e: No bootstrap required, opened a new log
I20260923 18:01:04.580135 25707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7401be5ddd2a49fda0cf26d121e0433e" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44599 } }
I20260923 18:01:04.580341 25707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:01:04.580374 25707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7401be5ddd2a49fda0cf26d121e0433e, State: Initialized, Role: FOLLOWER
I20260923 18:01:04.580772 25707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7401be5ddd2a49fda0cf26d121e0433e" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44599 } }
I20260923 18:01:04.580893 25707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:01:04.580940 25707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:01:04.581012 25707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:01:04.581810 25707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7401be5ddd2a49fda0cf26d121e0433e" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44599 } }
I20260923 18:01:04.582091 25707 leader_election.cc:302] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7401be5ddd2a49fda0cf26d121e0433e; no voters:
I20260923 18:01:04.582332 25707 leader_election.cc:288] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:01:04.582442 25712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:01:04.582743 25712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [term 1 LEADER]: Becoming Leader. State: Replica: 7401be5ddd2a49fda0cf26d121e0433e, State: Running, Role: LEADER
I20260923 18:01:04.583065 25712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7401be5ddd2a49fda0cf26d121e0433e" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44599 } }
I20260923 18:01:04.583710 25707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:01:04.584806 25713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7401be5ddd2a49fda0cf26d121e0433e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7401be5ddd2a49fda0cf26d121e0433e" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44599 } } }
I20260923 18:01:04.584957 25713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [sys.catalog]: This master's current role is: LEADER
I20260923 18:01:04.584805 25714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7401be5ddd2a49fda0cf26d121e0433e. Latest consensus state: current_term: 1 leader_uuid: "7401be5ddd2a49fda0cf26d121e0433e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7401be5ddd2a49fda0cf26d121e0433e" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44599 } } }
I20260923 18:01:04.585126 25714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e [sys.catalog]: This master's current role is: LEADER
I20260923 18:01:04.585451 25721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:01:04.588039 25721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:01:04.593824 25721 catalog_manager.cc:1347] Generated new cluster ID: 73fa3e0f5767425797fc456b62f76ccf
I20260923 18:01:04.593904 25721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:01:04.616858 25721 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:01:04.617269 25721 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:01:04.627601 25721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7401be5ddd2a49fda0cf26d121e0433e: Generated new TSK 0
I20260923 18:01:04.628545 25721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:01:04.631704 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:44599
--builtin_ntp_servers=127.15.139.148:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 18:01:04.747534 25731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:04.747856 25731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:04.747939 25731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:04.752118 25731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:04.752208 25731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:04.752305 25731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:01:04.757381 25731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:44599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:04.758538 25731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:04.760005 25731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:04.769939 25736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:04.769933 25737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:04.769930 25739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:04.769960 25731 server_base.cc:1034] running on GCE node
I20260923 18:01:04.770700 25731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:04.771379 25731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:04.772567 25731 hybrid_clock.cc:648] HybridClock initialized: now 1790186464772551 us; error 30 us; skew 500 ppm
I20260923 18:01:04.774540 25731 webserver.cc:460] Webserver started at http://127.15.139.129:39513/ using document root <none> and password file <none>
I20260923 18:01:04.775101 25731 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:04.775164 25731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:04.775332 25731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:01:04.777529 25731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "aad34ce29a6f46db9170b6866c793a69"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "3d9753d26106c98fe3a49b897e1ea7ba"
server_key_iv: "22f61b13dd155ea8c1231ec1e7ad437e"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.778093 25731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "aad34ce29a6f46db9170b6866c793a69"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "3d9753d26106c98fe3a49b897e1ea7ba"
server_key_iv: "22f61b13dd155ea8c1231ec1e7ad437e"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.781876 25731 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:01:04.784538 25745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.785974 25731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:01:04.786145 25731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "aad34ce29a6f46db9170b6866c793a69"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "3d9753d26106c98fe3a49b897e1ea7ba"
server_key_iv: "22f61b13dd155ea8c1231ec1e7ad437e"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.786263 25731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:04.790853 25731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:04.791457 25731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:04.791669 25731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:04.792210 25731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:01:04.793107 25731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:01:04.793184 25731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.793252 25731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:01:04.793306 25731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.802404 25731 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:36115
I20260923 18:01:04.802424 25858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:36115 every 8 connection(s)
I20260923 18:01:04.803336 25731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:01:04.808349 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 25731
I20260923 18:01:04.808481 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:01:04.808775 15918 external_mini_cluster.cc:1428] Setting key 17bd79f84b2ce3a5c98eb1a354348d90
I20260923 18:01:04.810181 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:04.805288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:33479 (local address 127.15.139.190:44599)
0923 18:01:04.805445 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:04.805449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:04.806657 (+ 1208us) server_negotiation.cc:407] Connection header received
0923 18:01:04.808092 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:04.808095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:04.808141 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:04.808259 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:04.809728 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:04.809731 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:04.809733 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:01:04.809743 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:04.809763 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:01:04.809770 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:04.809835 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:04.810041 (+ 206us) server_negotiation.cc:299] Negotiation successful
0923 18:01:04.810079 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260923 18:01:04.811131 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:04.805756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44599 (local address 127.15.139.129:33479)
0923 18:01:04.806472 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:04.806553 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:04.807867 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:04.808464 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:04.808475 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:04.808628 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:04.809404 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:01:04.809406 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:04.809463 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:01:04.809580 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:01:04.809852 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:01:04.809860 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:01:04.809866 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 18:01:04.810044 (+ 178us) client_negotiation.cc:240] Negotiation successful
0923 18:01:04.810381 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":145,"threads_started":1}
I20260923 18:01:04.811161 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:44599
--builtin_ntp_servers=127.15.139.148:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260923 18:01:04.812359 25859 heartbeater.cc:347] Connected to a master server at 127.15.139.190:44599
I20260923 18:01:04.812635 25859 heartbeater.cc:464] Registering TS with master...
I20260923 18:01:04.813045 25859 heartbeater.cc:511] Master 127.15.139.190:44599 requested a full tablet report, sending...
I20260923 18:01:04.814599 25671 ts_manager.cc:194] Registered new tserver with Master: aad34ce29a6f46db9170b6866c793a69 (127.15.139.129:36115)
W20260923 18:01:04.922629 25862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:04.922901 25862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:04.922941 25862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:04.926909 25862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:04.926990 25862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:04.927106 25862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:01:04.932054 25862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:44599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:04.933074 25862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:04.934382 25862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:04.943698 25870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:04.943781 25867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:04.943781 25868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:04.943931 25862 server_base.cc:1034] running on GCE node
I20260923 18:01:04.944602 25862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:04.945468 25862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:04.946789 25862 hybrid_clock.cc:648] HybridClock initialized: now 1790186464946752 us; error 51 us; skew 500 ppm
I20260923 18:01:04.948719 25862 webserver.cc:460] Webserver started at http://127.15.139.130:43611/ using document root <none> and password file <none>
I20260923 18:01:04.949312 25862 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:04.949411 25862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:04.949653 25862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:01:04.951834 25862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "5195792aff08498bb57b4a89454a69ae"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "08895d132ce3cb414c9dee906388b33e"
server_key_iv: "d11482d8e167e2080b082775431a77ba"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.952486 25862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "5195792aff08498bb57b4a89454a69ae"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "08895d132ce3cb414c9dee906388b33e"
server_key_iv: "d11482d8e167e2080b082775431a77ba"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.956267 25862 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260923 18:01:04.958906 25876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.960250 25862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:01:04.960386 25862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "5195792aff08498bb57b4a89454a69ae"
format_stamp: "Formatted at 2026-09-23 18:01:04 on dist-test-slave-vr57"
server_key: "08895d132ce3cb414c9dee906388b33e"
server_key_iv: "d11482d8e167e2080b082775431a77ba"
server_key_version: "encryptionkey@0"
I20260923 18:01:04.960508 25862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:04.965096 25862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:04.965683 25862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:04.965863 25862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:04.966389 25862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:01:04.967293 25862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:01:04.967363 25862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.967437 25862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:01:04.967491 25862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:04.976364 25862 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:36799
I20260923 18:01:04.976400 25989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:36799 every 8 connection(s)
I20260923 18:01:04.977350 25862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:01:04.978065 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 25862
I20260923 18:01:04.978209 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:01:04.978574 15918 external_mini_cluster.cc:1428] Setting key 22a3773906c9e16b66b7c4ba49a29914
I20260923 18:01:04.981451 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:44599
--builtin_ntp_servers=127.15.139.148:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260923 18:01:04.987435 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:04.981022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:52679 (local address 127.15.139.190:44599)
0923 18:01:04.981147 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:04.981152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:04.983488 (+ 2336us) server_negotiation.cc:407] Connection header received
0923 18:01:04.985481 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:04.985488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:04.985539 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:04.985684 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:04.986898 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:04.986904 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:04.986908 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:01:04.986925 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:04.986951 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:01:04.986964 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:04.987084 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:04.987190 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 18:01:04.987241 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 18:01:04.988318 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:04.979981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44599 (local address 127.15.139.130:52679)
0923 18:01:04.983305 (+ 3324us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:04.983368 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:04.985221 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:04.985849 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:04.985856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:04.985958 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:04.986606 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:01:04.986608 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:04.986654 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:01:04.986736 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:01:04.987072 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:01:04.987079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:01:04.987083 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 18:01:04.987225 (+ 142us) client_negotiation.cc:240] Negotiation successful
0923 18:01:04.987426 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":146,"threads_started":1}
I20260923 18:01:04.989682 25990 heartbeater.cc:347] Connected to a master server at 127.15.139.190:44599
I20260923 18:01:04.989920 25990 heartbeater.cc:464] Registering TS with master...
I20260923 18:01:04.990437 25990 heartbeater.cc:511] Master 127.15.139.190:44599 requested a full tablet report, sending...
I20260923 18:01:04.991593 25671 ts_manager.cc:194] Registered new tserver with Master: 5195792aff08498bb57b4a89454a69ae (127.15.139.130:36799)
W20260923 18:01:05.107172 25993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:05.107477 25993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:05.107586 25993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:05.111793 25993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:05.111865 25993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:05.111927 25993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:01:05.116796 25993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:44599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:05.118006 25993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:05.119414 25993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:05.129225 25999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:05.129244 25998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:05.129408 26001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:05.129489 25993 server_base.cc:1034] running on GCE node
I20260923 18:01:05.130146 25993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:05.130826 25993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:05.131984 25993 hybrid_clock.cc:648] HybridClock initialized: now 1790186465131962 us; error 41 us; skew 500 ppm
I20260923 18:01:05.133908 25993 webserver.cc:460] Webserver started at http://127.15.139.131:43635/ using document root <none> and password file <none>
I20260923 18:01:05.134505 25993 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:05.134569 25993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:05.134795 25993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:01:05.136996 25993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "f9e2adea06814deb91aea204570d8d90"
format_stamp: "Formatted at 2026-09-23 18:01:05 on dist-test-slave-vr57"
server_key: "675e236b65e2d7e5923546ad786e8658"
server_key_iv: "c3d0796fc457117da8324a8fab087c75"
server_key_version: "encryptionkey@0"
I20260923 18:01:05.137636 25993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "f9e2adea06814deb91aea204570d8d90"
format_stamp: "Formatted at 2026-09-23 18:01:05 on dist-test-slave-vr57"
server_key: "675e236b65e2d7e5923546ad786e8658"
server_key_iv: "c3d0796fc457117da8324a8fab087c75"
server_key_version: "encryptionkey@0"
I20260923 18:01:05.141614 25993 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:01:05.144498 26007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:05.145931 25993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:01:05.146098 25993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "f9e2adea06814deb91aea204570d8d90"
format_stamp: "Formatted at 2026-09-23 18:01:05 on dist-test-slave-vr57"
server_key: "675e236b65e2d7e5923546ad786e8658"
server_key_iv: "c3d0796fc457117da8324a8fab087c75"
server_key_version: "encryptionkey@0"
I20260923 18:01:05.146231 25993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:05.151100 25993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:05.151786 25993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:05.151988 25993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:05.152604 25993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:01:05.153558 25993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:01:05.153630 25993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:05.153703 25993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:01:05.153754 25993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:05.162897 25993 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:39427
I20260923 18:01:05.162915 26120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:39427 every 8 connection(s)
I20260923 18:01:05.163873 25993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:01:05.169212 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 25993
I20260923 18:01:05.169337 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:01:05.169697 15918 external_mini_cluster.cc:1428] Setting key 4d7409414fc8fdcfb81f6c875244ac72
I20260923 18:01:05.170454 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:05.165735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:40345 (local address 127.15.139.190:44599)
0923 18:01:05.165903 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:05.165908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:05.167130 (+ 1222us) server_negotiation.cc:407] Connection header received
0923 18:01:05.168504 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:05.168507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:05.168548 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:05.168648 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:05.169914 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:05.169918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:05.169921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:01:05.169934 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:05.169957 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:01:05.169972 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:05.170088 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:05.170271 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 18:01:05.170323 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 18:01:05.171576 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:05.166134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44599 (local address 127.15.139.131:40345)
0923 18:01:05.166875 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:05.166929 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:05.168287 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:05.168804 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:05.168810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:05.168895 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:05.169581 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:01:05.169582 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:05.169621 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:01:05.169686 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:01:05.170091 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:01:05.170100 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:01:05.170105 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 18:01:05.170313 (+ 208us) client_negotiation.cc:240] Negotiation successful
0923 18:01:05.170597 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":139,"threads_started":1}
I20260923 18:01:05.172722 26121 heartbeater.cc:347] Connected to a master server at 127.15.139.190:44599
I20260923 18:01:05.172982 26121 heartbeater.cc:464] Registering TS with master...
I20260923 18:01:05.173396 26121 heartbeater.cc:511] Master 127.15.139.190:44599 requested a full tablet report, sending...
I20260923 18:01:05.174400 25671 ts_manager.cc:194] Registered new tserver with Master: f9e2adea06814deb91aea204570d8d90 (127.15.139.131:39427)
I20260923 18:01:05.183833 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 18:01:05.186955 25710 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:01:05.185937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.15.139.190:44599)
0923 18:01:05.186164 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:05.186170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:05.186267 (+ 97us) server_negotiation.cc:407] Connection header received
0923 18:01:05.186424 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:05.186427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:05.186465 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:05.186563 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:05.186840 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50068: BlockingRecv error: recv got EOF from 127.0.0.1:50068 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
I20260923 18:01:05.187718 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 25731
I20260923 18:01:05.192996 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 25862
I20260923 18:01:05.198338 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 25993
I20260923 18:01:05.203629 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 25640
2026-09-23T18:01:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (833 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:01:05 dist-test-slave-vr57 krb5kdc[26131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:01:05 dist-test-slave-vr57 krb5kdc[26131](info): set up 2 sockets
Sep 23 18:01:05 dist-test-slave-vr57 krb5kdc[26131](info): commencing operation
krb5kdc: starting...
W20260923 18:01:06.269134 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:01:06 dist-test-slave-vr57 krb5kdc[26131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:01:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:01:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:01:06.399806 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40651
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:40651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:01:06.515035 26144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:06.515424 26144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:06.515537 26144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:06.519927 26144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:01:06.520049 26144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:06.520084 26144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:06.520123 26144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:01:06.520159 26144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:01:06.526002 26144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:40651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:06.527346 26144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:06.528616 26144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:06.538033 26152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:06.538033 26149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:06.538056 26150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:06.538383 26144 server_base.cc:1034] running on GCE node
I20260923 18:01:06.539070 26144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:06.540382 26144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:06.541586 26144 hybrid_clock.cc:648] HybridClock initialized: now 1790186466541573 us; error 31 us; skew 500 ppm
Sep 23 18:01:06 dist-test-slave-vr57 krb5kdc[26131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186466, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:01:06.544669 26144 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:01:06.545656 26144 webserver.cc:460] Webserver started at http://127.15.139.190:34745/ using document root <none> and password file <none>
I20260923 18:01:06.546264 26144 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:06.546357 26144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:06.546597 26144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:01:06.548786 26144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "fafa7b647b49439aa8e39a8650b57c0f"
format_stamp: "Formatted at 2026-09-23 18:01:06 on dist-test-slave-vr57"
server_key: "7a0d58757820ef2959e1a03bca0739d3"
server_key_iv: "23c1d0e2195246cf0e818bb369e1a22a"
server_key_version: "encryptionkey@0"
I20260923 18:01:06.549356 26144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "fafa7b647b49439aa8e39a8650b57c0f"
format_stamp: "Formatted at 2026-09-23 18:01:06 on dist-test-slave-vr57"
server_key: "7a0d58757820ef2959e1a03bca0739d3"
server_key_iv: "23c1d0e2195246cf0e818bb369e1a22a"
server_key_version: "encryptionkey@0"
I20260923 18:01:06.553508 26144 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:01:06.556174 26159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:06.557618 26144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:01:06.557816 26144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "fafa7b647b49439aa8e39a8650b57c0f"
format_stamp: "Formatted at 2026-09-23 18:01:06 on dist-test-slave-vr57"
server_key: "7a0d58757820ef2959e1a03bca0739d3"
server_key_iv: "23c1d0e2195246cf0e818bb369e1a22a"
server_key_version: "encryptionkey@0"
I20260923 18:01:06.557946 26144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:06.574025 26144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:06.577229 26144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:06.577446 26144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:06.585211 26144 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:40651
I20260923 18:01:06.585522 26211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:40651 every 8 connection(s)
I20260923 18:01:06.586316 26144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:01:06.586866 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 26144
I20260923 18:01:06.587006 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:01:06.587287 15918 external_mini_cluster.cc:1428] Setting key 5027725f520ac50373cb8a11e02d13f9
I20260923 18:01:06.589911 26212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:01:06.595969 26212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f: Bootstrap starting.
Sep 23 18:01:06 dist-test-slave-vr57 krb5kdc[26131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:01:06.598686 26212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f: Neither blocks nor log segments found. Creating new log.
I20260923 18:01:06.599606 26212 log.cc:824] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f: Log is configured to *not* fsync() on all Append() calls
I20260923 18:01:06.601768 26212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f: No bootstrap required, opened a new log
I20260923 18:01:06.602613 26215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:06.588892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45708 (local address 127.15.139.190:40651)
0923 18:01:06.589392 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:06.589412 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:06.589456 (+ 44us) server_negotiation.cc:407] Connection header received
0923 18:01:06.590212 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:06.590236 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:06.590579 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:06.591022 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:06.591844 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:06.592867 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:06.593786 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:06.594269 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:06.594452 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:06.597395 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:06.597401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:06.597420 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:01:06.597452 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:06.599346 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:06.599887 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:06.599893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:06.599900 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:06.600080 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:06.600452 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:06.600455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:06.600457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:06.600907 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:06.601151 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:06.601462 (+ 311us) server_negotiation.cc:299] Negotiation successful
0923 18:01:06.601785 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":179,"threads_started":1}
I20260923 18:01:06.605959 26212 raft_consensus.cc:348] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fafa7b647b49439aa8e39a8650b57c0f" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40651 } }
I20260923 18:01:06.606209 26212 raft_consensus.cc:374] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:01:06.606293 26212 raft_consensus.cc:729] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafa7b647b49439aa8e39a8650b57c0f, State: Initialized, Role: FOLLOWER
I20260923 18:01:06.606777 26212 consensus_queue.cc:260] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fafa7b647b49439aa8e39a8650b57c0f" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40651 } }
I20260923 18:01:06.606935 26212 raft_consensus.cc:388] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:01:06.607016 26212 raft_consensus.cc:482] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:01:06.607115 26212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:01:06.608139 26212 raft_consensus.cc:504] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fafa7b647b49439aa8e39a8650b57c0f" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40651 } }
I20260923 18:01:06.608489 26212 leader_election.cc:302] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fafa7b647b49439aa8e39a8650b57c0f; no voters:
I20260923 18:01:06.608817 26212 leader_election.cc:288] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:01:06.609004 26217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:01:06.609315 26217 raft_consensus.cc:686] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [term 1 LEADER]: Becoming Leader. State: Replica: fafa7b647b49439aa8e39a8650b57c0f, State: Running, Role: LEADER
I20260923 18:01:06.610005 26217 consensus_queue.cc:237] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fafa7b647b49439aa8e39a8650b57c0f" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40651 } }
I20260923 18:01:06.610185 26212 sys_catalog.cc:564] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:01:06.611770 26218 sys_catalog.cc:455] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fafa7b647b49439aa8e39a8650b57c0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fafa7b647b49439aa8e39a8650b57c0f" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40651 } } }
I20260923 18:01:06.611912 26218 sys_catalog.cc:458] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [sys.catalog]: This master's current role is: LEADER
I20260923 18:01:06.612228 26219 sys_catalog.cc:455] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fafa7b647b49439aa8e39a8650b57c0f. Latest consensus state: current_term: 1 leader_uuid: "fafa7b647b49439aa8e39a8650b57c0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fafa7b647b49439aa8e39a8650b57c0f" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40651 } } }
I20260923 18:01:06.612316 26219 sys_catalog.cc:458] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f [sys.catalog]: This master's current role is: LEADER
W20260923 18:01:06.614789 26232 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:01:06.614853 26232 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:01:06.615088 26229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:01:06.615927 26229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:01:06.620803 26229 catalog_manager.cc:1347] Generated new cluster ID: 0db916f911d2490ba37475b9071f4db8
I20260923 18:01:06.620874 26229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:01:06.644526 26229 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:01:06.645509 26229 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:01:06.650321 26229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f: Generated new TSK 0
I20260923 18:01:06.651026 26229 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:01:06.687953 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40651
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:01:06.799422 26238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:06.799772 26238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:06.799816 26238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:06.803854 26238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:06.803907 26238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:06.803963 26238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:01:06.809146 26238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:06.810184 26238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:06.811251 26238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:06.821401 26243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:06.821411 26244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:06.821389 26246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:06.821672 26238 server_base.cc:1034] running on GCE node
I20260923 18:01:06.822213 26238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:06.822921 26238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:06.824156 26238 hybrid_clock.cc:648] HybridClock initialized: now 1790186466824125 us; error 52 us; skew 500 ppm
Sep 23 18:01:06 dist-test-slave-vr57 krb5kdc[26131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186466, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:01:06.827015 26238 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:01:06.827970 26238 webserver.cc:460] Webserver started at http://127.15.139.129:40501/ using document root <none> and password file <none>
I20260923 18:01:06.828537 26238 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:06.828634 26238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:06.828876 26238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:01:06.831029 26238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "6037542acd5f40d0820ab495b9b6bbc9"
format_stamp: "Formatted at 2026-09-23 18:01:06 on dist-test-slave-vr57"
server_key: "6db5e30e1728729ca2db15ea84545bfe"
server_key_iv: "5516bd6dab28ceee0cfe092796b553ec"
server_key_version: "encryptionkey@0"
I20260923 18:01:06.831673 26238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "6037542acd5f40d0820ab495b9b6bbc9"
format_stamp: "Formatted at 2026-09-23 18:01:06 on dist-test-slave-vr57"
server_key: "6db5e30e1728729ca2db15ea84545bfe"
server_key_iv: "5516bd6dab28ceee0cfe092796b553ec"
server_key_version: "encryptionkey@0"
I20260923 18:01:06.835448 26238 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:01:06.837927 26253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:06.839183 26238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:01:06.839321 26238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "6037542acd5f40d0820ab495b9b6bbc9"
format_stamp: "Formatted at 2026-09-23 18:01:06 on dist-test-slave-vr57"
server_key: "6db5e30e1728729ca2db15ea84545bfe"
server_key_iv: "5516bd6dab28ceee0cfe092796b553ec"
server_key_version: "encryptionkey@0"
I20260923 18:01:06.839438 26238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:06.852451 26238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:06.855857 26238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:06.856084 26238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:06.856683 26238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:01:06.857542 26238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:01:06.857614 26238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:06.857687 26238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:01:06.857730 26238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:06.867471 26238 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:45501
I20260923 18:01:06.867587 26366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:45501 every 8 connection(s)
I20260923 18:01:06.868459 26238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:01:06.874233 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 26238
I20260923 18:01:06.874333 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:01:06.874614 15918 external_mini_cluster.cc:1428] Setting key 479fc9243d0258b688f13fc0ae7e71d4
Sep 23 18:01:06 dist-test-slave-vr57 krb5kdc[26131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186466, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:01:06.883644 26215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:06.870287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:55611 (local address 127.15.139.190:40651)
0923 18:01:06.870458 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:06.870463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:06.871332 (+ 869us) server_negotiation.cc:407] Connection header received
0923 18:01:06.872745 (+ 1413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:06.872748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:06.872796 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:06.872873 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:06.874122 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:06.874862 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:06.876365 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:06.876613 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:06.876697 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:06.879482 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:06.879489 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:06.879495 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:01:06.879567 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:06.881359 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:06.881902 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:06.881909 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:06.881912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:06.882015 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:06.882351 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:06.882355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:06.882359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:06.882626 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:06.882750 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:06.883239 (+ 489us) server_negotiation.cc:299] Negotiation successful
0923 18:01:06.883385 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260923 18:01:06.884387 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:06.870710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40651 (local address 127.15.139.129:55611)
0923 18:01:06.871171 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:06.871215 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:06.872524 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:06.873035 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:06.873042 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:06.873396 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:06.873892 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:06.873905 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:06.875001 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:06.875012 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:06.876228 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:06.876240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:06.876774 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:06.876781 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:06.877020 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:06.877704 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:06.877731 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:06.879300 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:01:06.881493 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:01:06.881498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:01:06.881512 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:01:06.881795 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:01:06.882117 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:01:06.882120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:01:06.882121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:01:06.882254 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:01:06.882746 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:01:06.882751 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:01:06.882988 (+ 237us) client_negotiation.cc:769] Sending connection context
0923 18:01:06.883153 (+ 165us) client_negotiation.cc:240] Negotiation successful
0923 18:01:06.883410 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
I20260923 18:01:06.885704 26367 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40651
I20260923 18:01:06.885982 26367 heartbeater.cc:464] Registering TS with master...
I20260923 18:01:06.886504 26367 heartbeater.cc:511] Master 127.15.139.190:40651 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:01:06.888201 26176 ts_manager.cc:194] Registered new tserver with Master: 6037542acd5f40d0820ab495b9b6bbc9 (127.15.139.129:45501)
I20260923 18:01:06.889853 26176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:55611
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:01:06.903153 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40651
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:01:07.019119 26372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:07.019409 26372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:07.019487 26372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:07.023610 26372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:07.023701 26372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:07.023789 26372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:01:07.029212 26372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:07.030337 26372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:07.031391 26372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:07.041136 26378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:07.041074 26377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:07.041185 26372 server_base.cc:1034] running on GCE node
W20260923 18:01:07.041325 26380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:07.041702 26372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:07.042465 26372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:07.043638 26372 hybrid_clock.cc:648] HybridClock initialized: now 1790186467043622 us; error 29 us; skew 500 ppm
Sep 23 18:01:07 dist-test-slave-vr57 krb5kdc[26131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186467, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:01:07.046496 26372 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:01:07.047466 26372 webserver.cc:460] Webserver started at http://127.15.139.130:43693/ using document root <none> and password file <none>
I20260923 18:01:07.048084 26372 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:07.048146 26372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:07.048381 26372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:01:07.050426 26372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "bc5136c960d345b9adb98f71743d5f06"
format_stamp: "Formatted at 2026-09-23 18:01:07 on dist-test-slave-vr57"
server_key: "36f51bce07722b45f407898f63b82cba"
server_key_iv: "88bb58c729aa8a417feffbaa73dfa84f"
server_key_version: "encryptionkey@0"
I20260923 18:01:07.051012 26372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "bc5136c960d345b9adb98f71743d5f06"
format_stamp: "Formatted at 2026-09-23 18:01:07 on dist-test-slave-vr57"
server_key: "36f51bce07722b45f407898f63b82cba"
server_key_iv: "88bb58c729aa8a417feffbaa73dfa84f"
server_key_version: "encryptionkey@0"
I20260923 18:01:07.054785 26372 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260923 18:01:07.057274 26387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:07.058446 26372 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:01:07.058612 26372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "bc5136c960d345b9adb98f71743d5f06"
format_stamp: "Formatted at 2026-09-23 18:01:07 on dist-test-slave-vr57"
server_key: "36f51bce07722b45f407898f63b82cba"
server_key_iv: "88bb58c729aa8a417feffbaa73dfa84f"
server_key_version: "encryptionkey@0"
I20260923 18:01:07.058724 26372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:07.082629 26372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:07.086202 26372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:07.086464 26372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:07.087184 26372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:01:07.088476 26372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:01:07.088568 26372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:07.088644 26372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:01:07.088702 26372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:07.098419 26372 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:41245
I20260923 18:01:07.098441 26500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:41245 every 8 connection(s)
I20260923 18:01:07.099458 26372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:01:07.099879 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 26372
I20260923 18:01:07.100018 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:01:07.100420 15918 external_mini_cluster.cc:1428] Setting key 1cdf31e42d58016fde2da3a549920690
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
Sep 23 18:01:07 dist-test-slave-vr57 krb5kdc[26131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186467, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:01:07.120922 26215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.101643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:44357 (local address 127.15.139.190:40651)
0923 18:01:07.101787 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:07.101792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:07.102845 (+ 1053us) server_negotiation.cc:407] Connection header received
0923 18:01:07.105028 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:07.105031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:07.105080 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:07.105161 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:07.106903 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.107455 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.109118 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.109415 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.109519 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.116273 (+ 6754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:07.116281 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:07.116287 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:01:07.116330 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:07.118411 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:07.119021 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:07.119025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:07.119027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:07.119123 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:07.119601 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:07.119605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:07.119607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:07.119830 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:07.119950 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:07.120582 (+ 632us) server_negotiation.cc:299] Negotiation successful
0923 18:01:07.120735 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 18:01:07.121801 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.102186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40651 (local address 127.15.139.130:44357)
0923 18:01:07.102694 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:07.102749 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:07.104753 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:07.105383 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:07.105395 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:07.105955 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:07.106709 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.106735 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.107681 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.107684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.108910 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.108938 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.109722 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.109726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.109965 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.110833 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:07.110860 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:07.116012 (+ 5152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:01:07.118552 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:01:07.118557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:01:07.118575 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:01:07.118886 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:01:07.119227 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:01:07.119233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:01:07.119237 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:01:07.119450 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:01:07.119950 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:01:07.119959 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:01:07.120302 (+ 343us) client_negotiation.cc:769] Sending connection context
0923 18:01:07.120513 (+ 211us) client_negotiation.cc:240] Negotiation successful
0923 18:01:07.120892 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":134,"threads_started":1}
I20260923 18:01:07.123117 26501 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40651
I20260923 18:01:07.123409 26501 heartbeater.cc:464] Registering TS with master...
I20260923 18:01:07.123986 26501 heartbeater.cc:511] Master 127.15.139.190:40651 requested a full tablet report, sending...
I20260923 18:01:07.125226 26176 ts_manager.cc:194] Registered new tserver with Master: bc5136c960d345b9adb98f71743d5f06 (127.15.139.130:41245)
I20260923 18:01:07.125851 26176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:44357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:01:07.134625 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40651
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:01:07.249220 26506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:07.249519 26506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:07.249565 26506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:07.253721 26506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:07.253791 26506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:07.253888 26506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:01:07.259310 26506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:07.260474 26506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:07.261888 26506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:07.271991 26514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:07.271986 26511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:07.271989 26512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:07.272400 26506 server_base.cc:1034] running on GCE node
I20260923 18:01:07.272817 26506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:07.273465 26506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:07.274658 26506 hybrid_clock.cc:648] HybridClock initialized: now 1790186467274642 us; error 30 us; skew 500 ppm
Sep 23 18:01:07 dist-test-slave-vr57 krb5kdc[26131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186467, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:01:07.277566 26506 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:01:07.278517 26506 webserver.cc:460] Webserver started at http://127.15.139.131:43271/ using document root <none> and password file <none>
I20260923 18:01:07.279093 26506 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:07.279217 26506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:07.279455 26506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:01:07.281540 26506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "d0e8922186e54b3fb3f9573371a004dc"
format_stamp: "Formatted at 2026-09-23 18:01:07 on dist-test-slave-vr57"
server_key: "2d9e5be7c47e87e20e662e93bdf7a8a4"
server_key_iv: "1b972b288c4bc8b96f989be7b980d087"
server_key_version: "encryptionkey@0"
I20260923 18:01:07.282117 26506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "d0e8922186e54b3fb3f9573371a004dc"
format_stamp: "Formatted at 2026-09-23 18:01:07 on dist-test-slave-vr57"
server_key: "2d9e5be7c47e87e20e662e93bdf7a8a4"
server_key_iv: "1b972b288c4bc8b96f989be7b980d087"
server_key_version: "encryptionkey@0"
I20260923 18:01:07.285869 26506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260923 18:01:07.288388 26521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:07.289604 26506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:01:07.289734 26506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "d0e8922186e54b3fb3f9573371a004dc"
format_stamp: "Formatted at 2026-09-23 18:01:07 on dist-test-slave-vr57"
server_key: "2d9e5be7c47e87e20e662e93bdf7a8a4"
server_key_iv: "1b972b288c4bc8b96f989be7b980d087"
server_key_version: "encryptionkey@0"
I20260923 18:01:07.289842 26506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:07.305091 26506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:07.308604 26506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:07.308823 26506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:07.309432 26506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:01:07.310322 26506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:01:07.310393 26506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:07.310457 26506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:01:07.310501 26506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:07.320367 26506 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:45637
I20260923 18:01:07.320390 26634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:45637 every 8 connection(s)
I20260923 18:01:07.321403 26506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:01:07.331801 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 26506
I20260923 18:01:07.331907 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
Sep 23 18:01:07 dist-test-slave-vr57 krb5kdc[26131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186467, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:01:07.332322 15918 external_mini_cluster.cc:1428] Setting key 07b471cdee54adc8244c04b997dd828e
I20260923 18:01:07.336643 26215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.323478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:41281 (local address 127.15.139.190:40651)
0923 18:01:07.323653 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:07.323657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:07.324593 (+ 936us) server_negotiation.cc:407] Connection header received
0923 18:01:07.326174 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:07.326178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:07.326238 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:07.326330 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:07.327623 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.328169 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.329510 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.329743 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.329859 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.332784 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:07.332788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:07.332791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:01:07.332828 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:07.334381 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:07.335038 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:07.335041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:07.335043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:07.335128 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:07.335489 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:07.335493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:07.335494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:07.335702 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:07.335811 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:07.336329 (+ 518us) server_negotiation.cc:299] Negotiation successful
0923 18:01:07.336455 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260923 18:01:07.337481 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.323953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40651 (local address 127.15.139.131:41281)
0923 18:01:07.324426 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:07.324474 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:07.325831 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:07.326530 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:07.326537 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:07.326934 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:07.327444 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.327458 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.328301 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.328305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.329377 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.329387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.329903 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.329907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.330099 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.330757 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:07.330774 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:07.332531 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:01:07.334545 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:01:07.334552 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:01:07.334569 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:01:07.334919 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:01:07.335222 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:01:07.335226 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:01:07.335228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:01:07.335381 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:01:07.335809 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:01:07.335817 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:01:07.336089 (+ 272us) client_negotiation.cc:769] Sending connection context
0923 18:01:07.336275 (+ 186us) client_negotiation.cc:240] Negotiation successful
0923 18:01:07.336617 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":110,"threads_started":1}
I20260923 18:01:07.338707 26635 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40651
I20260923 18:01:07.338933 26635 heartbeater.cc:464] Registering TS with master...
I20260923 18:01:07.339408 26635 heartbeater.cc:511] Master 127.15.139.190:40651 requested a full tablet report, sending...
I20260923 18:01:07.340750 26176 ts_manager.cc:194] Registered new tserver with Master: d0e8922186e54b3fb3f9573371a004dc (127.15.139.131:45637)
I20260923 18:01:07.341399 26176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:41281
I20260923 18:01:07.346271 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:01:07.356000 26215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.348298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.15.139.190:40651)
0923 18:01:07.348444 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:07.348448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:07.348656 (+ 208us) server_negotiation.cc:407] Connection header received
0923 18:01:07.348742 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:07.348744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:07.348786 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:07.348857 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:07.349534 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.350070 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.350902 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.351144 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.351225 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.352632 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:07.352635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:07.352637 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:01:07.352661 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:07.354196 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:07.354646 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:07.354648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:07.354650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:07.354759 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:07.355029 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:07.355031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:07.355032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:07.355239 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:07.355342 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:07.355656 (+ 314us) server_negotiation.cc:299] Negotiation successful
0923 18:01:07.355783 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 18:01:07.358803 26176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:01:07.362175 26176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:01:07.376758 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.370903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:45637 (local address 127.0.0.1:60216)
0923 18:01:07.372041 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:07.372055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:07.372143 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:07.372700 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:07.372706 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:07.372737 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:01:07.372921 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.372929 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.374017 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.374020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.375236 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.375246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.376200 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.376204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.376314 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.376357 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 18:01:07.376477 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 18:01:07.376599 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"mutex_wait_us":4,"thread_start_us":104,"threads_started":1}
I20260923 18:01:07.376783 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.371621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.15.139.131:45637)
0923 18:01:07.372127 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:07.372138 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:07.372160 (+ 22us) server_negotiation.cc:407] Connection header received
0923 18:01:07.372249 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:07.372255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:07.372456 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:07.372607 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:01:07.373062 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.373877 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.375382 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.376078 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.376218 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.376360 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:07.376524 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 18:01:07.376623 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":172,"threads_started":1}
I20260923 18:01:07.377422 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.371181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:45501 (local address 127.0.0.1:57122)
0923 18:01:07.372298 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:07.372310 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:07.372385 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:07.372893 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:07.372896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:07.372911 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:01:07.373038 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.373045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.374661 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.374664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.375916 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.375928 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.377040 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.377044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.377122 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.377144 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 18:01:07.377203 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 18:01:07.377275 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1066,"mutex_wait_us":5,"thread_start_us":101,"threads_started":1}
I20260923 18:01:07.377475 26650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.371476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57122 (local address 127.15.139.129:45501)
0923 18:01:07.371878 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:07.371895 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:07.372480 (+ 585us) server_negotiation.cc:407] Connection header received
0923 18:01:07.372562 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:07.372567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:07.372738 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:07.372920 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:01:07.373154 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.374455 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.376062 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.376907 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.377046 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.377179 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:07.377231 (+ 52us) server_negotiation.cc:299] Negotiation successful
0923 18:01:07.377326 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":91,"threads_started":1}
I20260923 18:01:07.377630 26646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.370826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:41245 (local address 127.0.0.1:44108)
0923 18:01:07.371673 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:07.371728 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:07.371891 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:07.372701 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:07.372707 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:07.372757 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:01:07.372979 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.372988 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.374604 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.374608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.375928 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.375934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.377258 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.377261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.377337 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.377355 (+ 18us) client_negotiation.cc:769] Sending connection context
0923 18:01:07.377419 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 18:01:07.377508 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":231,"spinlock_wait_cycles":31616,"thread_start_us":80,"threads_started":1}
I20260923 18:01:07.378088 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.370958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44108 (local address 127.15.139.130:41245)
0923 18:01:07.371880 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:07.371892 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:07.371919 (+ 27us) server_negotiation.cc:407] Connection header received
0923 18:01:07.372115 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:07.372122 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:07.372354 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:07.372504 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:01:07.373142 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.374444 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.376099 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.376950 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.377072 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.377200 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:07.377820 (+ 620us) server_negotiation.cc:299] Negotiation successful
0923 18:01:07.377920 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":796,"thread_start_us":114,"threads_started":1}
I20260923 18:01:07.380482 26569 tablet_service.cc:1467] Processing CreateTablet for tablet 5e8b96be28314e26bbba1c097993510d (DEFAULT_TABLE table=test-table [id=935573412de245d0948610b58ad75ee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:01:07.380558 26301 tablet_service.cc:1467] Processing CreateTablet for tablet 5e8b96be28314e26bbba1c097993510d (DEFAULT_TABLE table=test-table [id=935573412de245d0948610b58ad75ee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:01:07.380863 26435 tablet_service.cc:1467] Processing CreateTablet for tablet 5e8b96be28314e26bbba1c097993510d (DEFAULT_TABLE table=test-table [id=935573412de245d0948610b58ad75ee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:01:07.381597 26301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e8b96be28314e26bbba1c097993510d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:01:07.381608 26569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e8b96be28314e26bbba1c097993510d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:01:07.381850 26435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e8b96be28314e26bbba1c097993510d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:01:07.390298 26654 tablet_bootstrap.cc:492] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06: Bootstrap starting.
I20260923 18:01:07.390298 26653 tablet_bootstrap.cc:492] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9: Bootstrap starting.
I20260923 18:01:07.392767 26653 tablet_bootstrap.cc:654] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9: Neither blocks nor log segments found. Creating new log.
I20260923 18:01:07.393241 26654 tablet_bootstrap.cc:654] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06: Neither blocks nor log segments found. Creating new log.
I20260923 18:01:07.394155 26654 log.cc:824] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06: Log is configured to *not* fsync() on all Append() calls
I20260923 18:01:07.394214 26652 tablet_bootstrap.cc:492] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc: Bootstrap starting.
I20260923 18:01:07.396438 26654 tablet_bootstrap.cc:492] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06: No bootstrap required, opened a new log
I20260923 18:01:07.396631 26654 ts_tablet_manager.cc:1397] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260923 18:01:07.396672 26652 tablet_bootstrap.cc:654] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc: Neither blocks nor log segments found. Creating new log.
I20260923 18:01:07.397981 26652 log.cc:824] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc: Log is configured to *not* fsync() on all Append() calls
I20260923 18:01:07.398458 26653 log.cc:824] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9: Log is configured to *not* fsync() on all Append() calls
I20260923 18:01:07.400723 26652 tablet_bootstrap.cc:492] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc: No bootstrap required, opened a new log
I20260923 18:01:07.400943 26652 ts_tablet_manager.cc:1397] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260923 18:01:07.400878 26654 raft_consensus.cc:348] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc5136c960d345b9adb98f71743d5f06" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 41245 } } peers { permanent_uuid: "d0e8922186e54b3fb3f9573371a004dc" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45637 } } peers { permanent_uuid: "6037542acd5f40d0820ab495b9b6bbc9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 45501 } }
I20260923 18:01:07.401186 26654 raft_consensus.cc:374] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:01:07.401247 26654 raft_consensus.cc:729] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc5136c960d345b9adb98f71743d5f06, State: Initialized, Role: FOLLOWER
I20260923 18:01:07.401722 26654 consensus_queue.cc:260] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc5136c960d345b9adb98f71743d5f06" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 41245 } } peers { permanent_uuid: "d0e8922186e54b3fb3f9573371a004dc" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45637 } } peers { permanent_uuid: "6037542acd5f40d0820ab495b9b6bbc9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 45501 } }
I20260923 18:01:07.402424 26501 heartbeater.cc:503] Master 127.15.139.190:40651 was elected leader, sending a full tablet report...
I20260923 18:01:07.402773 26654 ts_tablet_manager.cc:1428] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260923 18:01:07.404737 26652 raft_consensus.cc:348] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc5136c960d345b9adb98f71743d5f06" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 41245 } } peers { permanent_uuid: "d0e8922186e54b3fb3f9573371a004dc" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45637 } } peers { permanent_uuid: "6037542acd5f40d0820ab495b9b6bbc9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 45501 } }
I20260923 18:01:07.405009 26652 raft_consensus.cc:374] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:01:07.405068 26652 raft_consensus.cc:729] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e8922186e54b3fb3f9573371a004dc, State: Initialized, Role: FOLLOWER
I20260923 18:01:07.405508 26653 tablet_bootstrap.cc:492] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9: No bootstrap required, opened a new log
I20260923 18:01:07.405511 26652 consensus_queue.cc:260] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc5136c960d345b9adb98f71743d5f06" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 41245 } } peers { permanent_uuid: "d0e8922186e54b3fb3f9573371a004dc" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45637 } } peers { permanent_uuid: "6037542acd5f40d0820ab495b9b6bbc9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 45501 } }
I20260923 18:01:07.405715 26653 ts_tablet_manager.cc:1397] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260923 18:01:07.406500 26652 ts_tablet_manager.cc:1428] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260923 18:01:07.406939 26635 heartbeater.cc:503] Master 127.15.139.190:40651 was elected leader, sending a full tablet report...
I20260923 18:01:07.410056 26653 raft_consensus.cc:348] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc5136c960d345b9adb98f71743d5f06" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 41245 } } peers { permanent_uuid: "d0e8922186e54b3fb3f9573371a004dc" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45637 } } peers { permanent_uuid: "6037542acd5f40d0820ab495b9b6bbc9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 45501 } }
I20260923 18:01:07.410341 26653 raft_consensus.cc:374] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:01:07.410399 26653 raft_consensus.cc:729] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6037542acd5f40d0820ab495b9b6bbc9, State: Initialized, Role: FOLLOWER
I20260923 18:01:07.410847 26653 consensus_queue.cc:260] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc5136c960d345b9adb98f71743d5f06" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 41245 } } peers { permanent_uuid: "d0e8922186e54b3fb3f9573371a004dc" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45637 } } peers { permanent_uuid: "6037542acd5f40d0820ab495b9b6bbc9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 45501 } }
I20260923 18:01:07.411605 26367 heartbeater.cc:503] Master 127.15.139.190:40651 was elected leader, sending a full tablet report...
I20260923 18:01:07.411895 26653 ts_tablet_manager.cc:1428] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260923 18:01:07.446879 26659 raft_consensus.cc:482] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:01:07.447080 26659 raft_consensus.cc:504] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc5136c960d345b9adb98f71743d5f06" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 41245 } } peers { permanent_uuid: "d0e8922186e54b3fb3f9573371a004dc" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45637 } } peers { permanent_uuid: "6037542acd5f40d0820ab495b9b6bbc9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 45501 } }
I20260923 18:01:07.448559 26659 leader_election.cc:288] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc5136c960d345b9adb98f71743d5f06 (127.15.139.130:41245), 6037542acd5f40d0820ab495b9b6bbc9 (127.15.139.129:45501)
I20260923 18:01:07.452759 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.448807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:45501 (local address 127.15.139.131:35097)
0923 18:01:07.448955 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:07.448973 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:07.449163 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:07.449612 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:07.449617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:07.449642 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:01:07.449773 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.449782 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.450691 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.450695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.451597 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.451606 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.452284 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.452291 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.452418 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.452441 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 18:01:07.452518 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 18:01:07.452576 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":4}
I20260923 18:01:07.452782 26650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.448944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:35097 (local address 127.15.139.129:45501)
0923 18:01:07.449070 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:07.449074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:07.449095 (+ 21us) server_negotiation.cc:407] Connection header received
0923 18:01:07.449360 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:07.449364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:07.449419 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:07.449519 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:01:07.449899 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.450580 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.451715 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.452163 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.452282 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.452364 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:07.452554 (+ 190us) server_negotiation.cc:299] Negotiation successful
0923 18:01:07.452658 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:01:07.452828 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.448816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:41245 (local address 127.15.139.131:53493)
0923 18:01:07.449459 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:07.449476 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:07.449609 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:07.449896 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:07.449899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:07.449916 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:01:07.450031 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.450037 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.450799 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.450802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.451698 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:07.451709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.452381 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.452387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:07.452529 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.452554 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 18:01:07.452631 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 18:01:07.452692 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":203,"thread_start_us":119,"threads_started":1}
I20260923 18:01:07.452865 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.448933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:53493 (local address 127.15.139.130:41245)
0923 18:01:07.449072 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:07.449077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:07.449641 (+ 564us) server_negotiation.cc:407] Connection header received
0923 18:01:07.449704 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:07.449707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:07.449761 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:07.449857 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:01:07.450158 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.450694 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.451816 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.452252 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.452383 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.452438 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:07.452674 (+ 236us) server_negotiation.cc:299] Negotiation successful
0923 18:01:07.452745 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 18:01:07.453478 26455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8b96be28314e26bbba1c097993510d" candidate_uuid: "d0e8922186e54b3fb3f9573371a004dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5136c960d345b9adb98f71743d5f06" is_pre_election: true
I20260923 18:01:07.453477 26321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8b96be28314e26bbba1c097993510d" candidate_uuid: "d0e8922186e54b3fb3f9573371a004dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6037542acd5f40d0820ab495b9b6bbc9" is_pre_election: true
I20260923 18:01:07.453835 26455 raft_consensus.cc:2454] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e8922186e54b3fb3f9573371a004dc in term 0.
I20260923 18:01:07.453835 26321 raft_consensus.cc:2454] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e8922186e54b3fb3f9573371a004dc in term 0.
I20260923 18:01:07.454380 26522 leader_election.cc:302] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6037542acd5f40d0820ab495b9b6bbc9, bc5136c960d345b9adb98f71743d5f06, d0e8922186e54b3fb3f9573371a004dc; no voters:
I20260923 18:01:07.454727 26659 raft_consensus.cc:2781] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:01:07.454828 26659 raft_consensus.cc:482] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:01:07.454883 26659 raft_consensus.cc:3037] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:01:07.456265 26659 raft_consensus.cc:504] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc5136c960d345b9adb98f71743d5f06" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 41245 } } peers { permanent_uuid: "d0e8922186e54b3fb3f9573371a004dc" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45637 } } peers { permanent_uuid: "6037542acd5f40d0820ab495b9b6bbc9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 45501 } }
I20260923 18:01:07.456645 26659 leader_election.cc:288] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [CANDIDATE]: Term 1 election: Requested vote from peers bc5136c960d345b9adb98f71743d5f06 (127.15.139.130:41245), 6037542acd5f40d0820ab495b9b6bbc9 (127.15.139.129:45501)
I20260923 18:01:07.456954 26455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8b96be28314e26bbba1c097993510d" candidate_uuid: "d0e8922186e54b3fb3f9573371a004dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5136c960d345b9adb98f71743d5f06"
I20260923 18:01:07.457108 26455 raft_consensus.cc:3037] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:01:07.457042 26321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8b96be28314e26bbba1c097993510d" candidate_uuid: "d0e8922186e54b3fb3f9573371a004dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6037542acd5f40d0820ab495b9b6bbc9"
I20260923 18:01:07.457261 26321 raft_consensus.cc:3037] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:01:07.458192 26455 raft_consensus.cc:2454] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e8922186e54b3fb3f9573371a004dc in term 1.
I20260923 18:01:07.458346 26321 raft_consensus.cc:2454] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e8922186e54b3fb3f9573371a004dc in term 1.
I20260923 18:01:07.458487 26522 leader_election.cc:302] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc5136c960d345b9adb98f71743d5f06, d0e8922186e54b3fb3f9573371a004dc; no voters:
I20260923 18:01:07.458673 26659 raft_consensus.cc:2781] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:01:07.458914 26659 raft_consensus.cc:686] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 1 LEADER]: Becoming Leader. State: Replica: d0e8922186e54b3fb3f9573371a004dc, State: Running, Role: LEADER
I20260923 18:01:07.459307 26659 consensus_queue.cc:237] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc5136c960d345b9adb98f71743d5f06" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 41245 } } peers { permanent_uuid: "d0e8922186e54b3fb3f9573371a004dc" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45637 } } peers { permanent_uuid: "6037542acd5f40d0820ab495b9b6bbc9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 45501 } }
I20260923 18:01:07.462745 26176 catalog_manager.cc:5495] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc reported cstate change: term changed from 0 to 1, leader changed from <none> to d0e8922186e54b3fb3f9573371a004dc (127.15.139.131). New cstate: current_term: 1 leader_uuid: "d0e8922186e54b3fb3f9573371a004dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc5136c960d345b9adb98f71743d5f06" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 41245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e8922186e54b3fb3f9573371a004dc" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6037542acd5f40d0820ab495b9b6bbc9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 45501 } health_report { overall_health: UNKNOWN } } }
I20260923 18:01:07.484356 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.480394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60232 (local address 127.15.139.131:45637)
0923 18:01:07.480566 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:07.480569 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:07.480585 (+ 16us) server_negotiation.cc:407] Connection header received
0923 18:01:07.480664 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:07.480666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:07.480724 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:07.480826 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:01:07.481162 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.481673 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.482562 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:07.482767 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:07.482846 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:07.483222 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:01:07.483738 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:01:07.483843 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:07.484045 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 18:01:07.484152 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260923 18:01:07.490635 26455 raft_consensus.cc:1261] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06 [term 1 FOLLOWER]: Refusing update from remote peer d0e8922186e54b3fb3f9573371a004dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:01:07.490635 26321 raft_consensus.cc:1261] T 5e8b96be28314e26bbba1c097993510d P 6037542acd5f40d0820ab495b9b6bbc9 [term 1 FOLLOWER]: Refusing update from remote peer d0e8922186e54b3fb3f9573371a004dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:01:07.491461 26662 consensus_queue.cc:1035] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc5136c960d345b9adb98f71743d5f06" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 41245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:01:07.491704 26659 consensus_queue.cc:1035] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6037542acd5f40d0820ab495b9b6bbc9" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 45501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:01:07.499900 26666 mvcc.cc:205] Tried to move back new op lower bound from 7332603770833629184 to 7332603770716635136. Current Snapshot: MvccSnapshot[applied={T|T < 7332603770833629184}]
I20260923 18:01:07.505213 26667 mvcc.cc:205] Tried to move back new op lower bound from 7332603770833629184 to 7332603770716635136. Current Snapshot: MvccSnapshot[applied={T|T < 7332603770833629184}]
I20260923 18:01:07.508021 26175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:01:07.508210 26175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:01:07.511029 26175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fafa7b647b49439aa8e39a8650b57c0f: Sending DeleteTablet for 3 replicas of tablet 5e8b96be28314e26bbba1c097993510d
I20260923 18:01:07.512193 26569 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e8b96be28314e26bbba1c097993510d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:01:07 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:60216
I20260923 18:01:07.512181 26301 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e8b96be28314e26bbba1c097993510d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:01:07 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:57122
I20260923 18:01:07.512681 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 26238
I20260923 18:01:07.512795 26435 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e8b96be28314e26bbba1c097993510d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:01:07 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:44108
I20260923 18:01:07.515139 26676 tablet_replica.cc:330] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06: stopping tablet replica
I20260923 18:01:07.515552 26676 raft_consensus.cc:2229] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:01:07.515928 26676 raft_consensus.cc:2258] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:01:07.517592 26676 ts_tablet_manager.cc:1905] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:01:07.519743 26675 tablet_replica.cc:330] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc: stopping tablet replica
I20260923 18:01:07.520041 26675 raft_consensus.cc:2229] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:01:07.520363 26676 ts_tablet_manager.cc:1918] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:01:07.520463 26676 log.cc:1196] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/5e8b96be28314e26bbba1c097993510d
I20260923 18:01:07.520455 26675 raft_consensus.cc:2258] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:01:07.520799 26676 ts_tablet_manager.cc:1939] T 5e8b96be28314e26bbba1c097993510d P bc5136c960d345b9adb98f71743d5f06: Deleting consensus metadata
I20260923 18:01:07.522064 26160 catalog_manager.cc:4841] TS bc5136c960d345b9adb98f71743d5f06 (127.15.139.130:41245): tablet 5e8b96be28314e26bbba1c097993510d (table test-table [id=935573412de245d0948610b58ad75ee7]) successfully deleted
I20260923 18:01:07.522387 26675 ts_tablet_manager.cc:1905] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:01:07.525870 26675 ts_tablet_manager.cc:1918] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:01:07.526006 26675 log.cc:1196] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/5e8b96be28314e26bbba1c097993510d
I20260923 18:01:07.526401 26675 ts_tablet_manager.cc:1939] T 5e8b96be28314e26bbba1c097993510d P d0e8922186e54b3fb3f9573371a004dc: Deleting consensus metadata
W20260923 18:01:07.526473 26161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:45501 (error 108)
I20260923 18:01:07.527415 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 26372
I20260923 18:01:07.527678 26646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:07.527290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:45501 (local address 127.0.0.1:57128)
0923 18:01:07.527449 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:07.527572 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260923 18:01:07.528563 26161 catalog_manager.cc:4584] TS 6037542acd5f40d0820ab495b9b6bbc9 (127.15.139.129:45501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e8b96be28314e26bbba1c097993510d: Network error: Client connection negotiation failed: client connection to 127.15.139.129:45501: connect: Connection refused (error 111)
I20260923 18:01:07.528890 26163 catalog_manager.cc:4841] TS d0e8922186e54b3fb3f9573371a004dc (127.15.139.131:45637): tablet 5e8b96be28314e26bbba1c097993510d (table test-table [id=935573412de245d0948610b58ad75ee7]) successfully deleted
I20260923 18:01:07.533407 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 26506
I20260923 18:01:07.539252 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 26144
2026-09-23T18:01:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2336 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:01:07 dist-test-slave-vr57 krb5kdc[26680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:01:07 dist-test-slave-vr57 krb5kdc[26680](info): set up 2 sockets
Sep 23 18:01:07 dist-test-slave-vr57 krb5kdc[26680](info): commencing operation
krb5kdc: starting...
W20260923 18:01:08.601711 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:01:08 dist-test-slave-vr57 krb5kdc[26680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:01:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:01:08Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.139.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.139.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/rangeradmin_127.15.139.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/rangeradmin_127.15.139.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.139.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.139.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/rangerlookup_127.15.139.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/rangerlookup_127.15.139.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.139.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.139.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/HTTP_127.15.139.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/HTTP_127.15.139.148@KRBTEST.COM.keytab.
I20260923 18:01:08.776546 15918 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwW7RpU/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/postgres -l logfile start
2026-09-23 18:01:11.286 UTC [26710] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 18:01:11.286 UTC [26710] LOG: listening on IPv4 address "127.15.139.148", port 46611
2026-09-23 18:01:11.290 UTC [26710] LOG: listening on Unix socket "/tmp/.s.PGSQL.46611"
2026-09-23 18:01:11.324 UTC [26713] LOG: database system was shut down at 2026-09-23 18:01:09 UTC
2026-09-23 18:01:11.328 UTC [26710] LOG: database system is ready to accept connections
I20260923 18:01:12.288818 15918 mini_postgres.cc:96] Postgres bound to 46611
2026-09-23 18:01:12.295 UTC [26721] FATAL: database "slave" does not exist
127.15.139.148:46611 - accepting connections
I20260923 18:01:12.296034 15918 mini_ranger.cc:161] Starting Ranger...
I20260923 18:01:12.312134 15918 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 18:01:12.796832 15918 mini_ranger.cc:87] Created ranger Postgres database
I20260923 18:01:12.796919 15918 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-admin
2026-09-23 18:01:13,146 [I] DB FLAVOR :POSTGRES
2026-09-23 18:01:13,146 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 18:01:13,146 [I] Checking connection..
2026-09-23 18:01:13,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 18:01:13,401 [I] Checking connection passed.
2026-09-23 18:01:13,401 [I] --------- Verifying version history table ---------
2026-09-23 18:01:13,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 18:01:13,666 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 18:01:13,666 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 18:01:13,910 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 18:01:13,910 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 18:01:13,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 18:01:14.163 UTC [26786] WARNING: there is no transaction in progress
2026-09-23 18:01:14,170 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 18:01:14,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 18:01:14,436 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 18:01:14,437 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 18:01:14,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 18:01:14,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 18:01:14,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vr57.c.gcp-upstream.internal','N') ;"
2026-09-23 18:01:15,194 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 18:01:15,476 [I] Table x_portal_user does not exist in database ranger
2026-09-23 18:01:15,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 18:01:15,766 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 18:01:15,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 18:01:16,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 18:01:16,288 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 18:01:16,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 18:01:17.278 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.295 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.312 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.329 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.344 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.361 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.427 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.434 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.448 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.459 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.469 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.483 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.493 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.500 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.507 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:17.513 UTC [26895] WARNING: there is no transaction in progress
2026-09-23 18:01:18,198 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 18:01:18,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 18:01:18,433 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 18:01:18,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 18:01:18,709 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 18:01:18,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 18:01:18,976 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 18:01:18,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 18:01:19,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 18:01:19,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vr57.c.gcp-upstream.internal';"
2026-09-23 18:01:19,736 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 18:01:19,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 18:01:19,985 [I] DB_PATCHES have already been applied
I20260923 18:01:19.990921 15918 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwW7RpU/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 18:01:19.991039 15918 mini_ranger.cc:193] Using host: 127.15.139.148
I20260923 18:01:19.994769 15918 mini_ranger.cc:227] Ranger admin URL: http://127.15.139.148:47515
Sep 23, 2026 6:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 6:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 6:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 6:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 6:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 6:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 6:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 18:01:21.108027 15918 mini_ranger.cc:160] Time spent starting Ranger: real 8.812s user 0.000s sys 0.011s
Sep 23, 2026 6:01:41 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 6:01:42 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 23, 2026 6:01:42 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 6:01:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 6:01:42 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 6:01:43 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 18:01:43.676759 15918 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.139.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.139.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/rangerkms_127.15.139.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/rangerkms_127.15.139.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.139.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.139.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.139.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/HTTP_127.15.139.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.139.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/HTTP_127.15.139.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 18:01:43 dist-test-slave-vr57 krb5kdc[26680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186503, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 18:01:43.768793 15918 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 18:01:43.779327 15918 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 18:01:44.671447 15918 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 18:01:44.671607 15918 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-kms
I20260923 18:01:44.671736 15918 mini_ranger_kms.cc:227] Using postgres at 127.15.139.148:46611
2026-09-23 18:01:44,860 [I] DB FLAVOR :POSTGRES
2026-09-23 18:01:44,861 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 18:01:44,861 [I] Checking connection
2026-09-23 18:01:44,861 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 18:01:45,119 [I] connection success
2026-09-23 18:01:45,119 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 18:01:45,119 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 18:01:45,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 18:01:45,362 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 18:01:45,362 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 18:01:45,362 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 18:01:45,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:46611/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 18:01:45,629 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 18:01:46.143915 15918 mini_ranger_kms.cc:310] Created kms service
I20260923 18:01:46.337486 15918 mini_ranger_kms.cc:326] Created kudu user
I20260923 18:01:46.384894 15918 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 18:01:46 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186503, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:01:46.777027 15918 mini_ranger_kms.cc:384] Added ranger policy
I20260923 18:01:46.777172 15918 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 18:01:46.777192 15918 mini_ranger_kms.cc:240] Using host: 127.15.139.148
I20260923 18:01:46.781529 15918 mini_ranger_kms.cc:276] Ranger KMS PID: 27077
I20260923 18:01:46.781661 15918 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.139.148:52691
Sep 23, 2026 6:01:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 6:01:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 6:01:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 6:01:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 6:01:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 6:01:48 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 18:01:49.094522 15918 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.326s user 0.015s sys 0.158s
I20260923 18:01:49.094671 15918 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 18:01:49.094817 15918 mini_ranger_kms.cc:401] 127.15.139.148:52691/kms/v1/keys
Sep 23 18:01:49 dist-test-slave-vr57 krb5kdc[26680](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790186509, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.139.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 18:01:49 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186509, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.139.148@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
Sep 23 18:01:50 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186509, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.139.148@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
Sep 23 18:01:52 dist-test-slave-vr57 krb5kdc[26680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:01:52.402822 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:45355
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:43193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:45355
--ranger_config_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.139.148:52691/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:01:52.520292 27128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:52.520630 27128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:52.520677 27128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:52.524914 27128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:01:52.524983 27128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:52.525002 27128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:52.525020 27128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:01:52.525035 27128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:01:52.530683 27128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:43193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.139.148:52691/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:45355
--ranger_config_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:45355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:52.531988 27128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:52.533195 27128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 18:01:52.542436 27128 server_base.cc:1034] running on GCE node
W20260923 18:01:52.542511 27134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:52.542484 27136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:52.542483 27133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:52.543311 27128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:52.544574 27128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:52.545780 27128 hybrid_clock.cc:648] HybridClock initialized: now 1790186512545762 us; error 33 us; skew 500 ppm
Sep 23 18:01:52 dist-test-slave-vr57 krb5kdc[26680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186512, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:01:52.548717 27128 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:01:52.549664 27128 webserver.cc:460] Webserver started at http://127.15.139.190:39143/ using document root <none> and password file <none>
I20260923 18:01:52.550172 27128 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:52.550227 27128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:52.550393 27128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 18:01:52 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186512, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:01:52.599396 27128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "f4cd225de1514279a3b0bb696f5e2c69"
format_stamp: "Formatted at 2026-09-23 18:01:52 on dist-test-slave-vr57"
server_key: "cd8bc65f624305b537559dfedd2c8add"
server_key_iv: "81cb850882e79ea07e8ad7b75b7ff8d2"
server_key_version: "kuduclusterkey@0"
I20260923 18:01:52.600273 27128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "f4cd225de1514279a3b0bb696f5e2c69"
format_stamp: "Formatted at 2026-09-23 18:01:52 on dist-test-slave-vr57"
server_key: "cd8bc65f624305b537559dfedd2c8add"
server_key_iv: "81cb850882e79ea07e8ad7b75b7ff8d2"
server_key_version: "kuduclusterkey@0"
I20260923 18:01:52.604624 27128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:01:52.619486 27143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:52.621117 27128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:01:52.621279 27128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "f4cd225de1514279a3b0bb696f5e2c69"
format_stamp: "Formatted at 2026-09-23 18:01:52 on dist-test-slave-vr57"
server_key: "cd8bc65f624305b537559dfedd2c8add"
server_key_iv: "81cb850882e79ea07e8ad7b75b7ff8d2"
server_key_version: "kuduclusterkey@0"
I20260923 18:01:52.621444 27128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:52.658895 27128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:52.662124 27128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:52.662348 27128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:52.669442 27128 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:45355
I20260923 18:01:52.669456 27196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:45355 every 8 connection(s)
I20260923 18:01:52.670408 27128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:01:52.670974 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 27128
I20260923 18:01:52.671123 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:01:52.671424 15918 external_mini_cluster.cc:1428] Setting key e7a1ec7548692f9f1d7fb7d4f706a0f7
I20260923 18:01:52.673504 27197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:01:52.680115 27197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69: Bootstrap starting.
Sep 23 18:01:52 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:01:52.682341 27197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69: Neither blocks nor log segments found. Creating new log.
I20260923 18:01:52.683431 27197 log.cc:824] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69: Log is configured to *not* fsync() on all Append() calls
I20260923 18:01:52.685770 27197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69: No bootstrap required, opened a new log
I20260923 18:01:52.685994 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:52.672880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46888 (local address 127.15.139.190:45355)
0923 18:01:52.673289 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:52.673309 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:52.673354 (+ 45us) server_negotiation.cc:407] Connection header received
0923 18:01:52.674020 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:52.674038 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:52.674357 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:52.674633 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:52.675594 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:52.676654 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:52.677814 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:52.678251 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:52.678363 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:52.681191 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:52.681197 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:52.681213 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:01:52.681244 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:52.682943 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:52.683403 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:52.683408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:52.683412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:52.683586 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:52.683929 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:52.683932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:52.683935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:52.684431 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:52.684581 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:52.684923 (+ 342us) server_negotiation.cc:299] Negotiation successful
0923 18:01:52.685215 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":155,"threads_started":1}
I20260923 18:01:52.689183 27197 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4cd225de1514279a3b0bb696f5e2c69" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 45355 } }
I20260923 18:01:52.689422 27197 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:01:52.689503 27197 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4cd225de1514279a3b0bb696f5e2c69, State: Initialized, Role: FOLLOWER
I20260923 18:01:52.689967 27197 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4cd225de1514279a3b0bb696f5e2c69" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 45355 } }
I20260923 18:01:52.690122 27197 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:01:52.690210 27197 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:01:52.690310 27197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:01:52.691213 27197 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4cd225de1514279a3b0bb696f5e2c69" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 45355 } }
I20260923 18:01:52.691641 27197 leader_election.cc:302] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4cd225de1514279a3b0bb696f5e2c69; no voters:
I20260923 18:01:52.691972 27197 leader_election.cc:288] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:01:52.692093 27202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:01:52.692366 27202 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [term 1 LEADER]: Becoming Leader. State: Replica: f4cd225de1514279a3b0bb696f5e2c69, State: Running, Role: LEADER
I20260923 18:01:52.692725 27202 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4cd225de1514279a3b0bb696f5e2c69" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 45355 } }
I20260923 18:01:52.693287 27197 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:01:52.694545 27203 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4cd225de1514279a3b0bb696f5e2c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4cd225de1514279a3b0bb696f5e2c69" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 45355 } } }
I20260923 18:01:52.694692 27203 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [sys.catalog]: This master's current role is: LEADER
I20260923 18:01:52.694545 27204 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4cd225de1514279a3b0bb696f5e2c69. Latest consensus state: current_term: 1 leader_uuid: "f4cd225de1514279a3b0bb696f5e2c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4cd225de1514279a3b0bb696f5e2c69" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 45355 } } }
I20260923 18:01:52.695051 27204 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69 [sys.catalog]: This master's current role is: LEADER
I20260923 18:01:52.695124 27211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:01:52.697855 27211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:01:52.698274 27197 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 18:01:52.707435 27211 catalog_manager.cc:1347] Generated new cluster ID: de6f1c2a5fb9454eb210bd25b45345c3
I20260923 18:01:52.707531 27211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:01:52.731310 27211 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:01:52.732766 27211 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:01:52.738471 27211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69: Generated new TSK 0
I20260923 18:01:52.739280 27211 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:01:53.443209 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.139.148:52691/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:45355
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:43193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:01:53.597291 27242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:53.597692 27242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:53.597772 27242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:53.603940 27242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:53.604034 27242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:53.604149 27242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:01:53.612538 27242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:43193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.139.148:52691/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:45355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:53.614020 27242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:53.615401 27242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:53.627107 27250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:53.627113 27249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:53.627094 27252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:53.627575 27242 server_base.cc:1034] running on GCE node
I20260923 18:01:53.628080 27242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:53.628921 27242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:53.630143 27242 hybrid_clock.cc:648] HybridClock initialized: now 1790186513630110 us; error 43 us; skew 500 ppm
Sep 23 18:01:53 dist-test-slave-vr57 krb5kdc[26680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186513, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:01:53.638643 27242 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:01:53.639810 27242 webserver.cc:460] Webserver started at http://127.15.139.129:42675/ using document root <none> and password file <none>
I20260923 18:01:53.640549 27242 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:53.640641 27242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:53.640911 27242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 18:01:53 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186513, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:01:53.658145 27242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "c8742513d1584075bf829459d2ff94bb"
format_stamp: "Formatted at 2026-09-23 18:01:53 on dist-test-slave-vr57"
server_key: "4ea4899eff054ca916164bfdeaec1076"
server_key_iv: "170e41a6f0641e6cf7b0973a1054e66b"
server_key_version: "kuduclusterkey@0"
I20260923 18:01:53.659006 27242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "c8742513d1584075bf829459d2ff94bb"
format_stamp: "Formatted at 2026-09-23 18:01:53 on dist-test-slave-vr57"
server_key: "4ea4899eff054ca916164bfdeaec1076"
server_key_iv: "170e41a6f0641e6cf7b0973a1054e66b"
server_key_version: "kuduclusterkey@0"
I20260923 18:01:53.664964 27242 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260923 18:01:53.687253 27260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:53.692507 27242 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.003s sys 0.000s
I20260923 18:01:53.692678 27242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "c8742513d1584075bf829459d2ff94bb"
format_stamp: "Formatted at 2026-09-23 18:01:53 on dist-test-slave-vr57"
server_key: "4ea4899eff054ca916164bfdeaec1076"
server_key_iv: "170e41a6f0641e6cf7b0973a1054e66b"
server_key_version: "kuduclusterkey@0"
I20260923 18:01:53.692834 27242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:53.716451 27242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:53.720245 27242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:53.720463 27242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:53.721186 27242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:01:53.722374 27242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:01:53.722442 27242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:53.722509 27242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:01:53.722543 27242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:53.733891 27242 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:42405
I20260923 18:01:53.735154 27242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:01:53.741048 27375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:42405 every 8 connection(s)
I20260923 18:01:53.743564 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 27242
I20260923 18:01:53.743687 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:01:53.744028 15918 external_mini_cluster.cc:1428] Setting key 648ea3b4d52f66833c3c61d7c0c63a5c
Sep 23 18:01:53 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186513, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:01:53.761034 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:53.739732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:34779 (local address 127.15.139.190:45355)
0923 18:01:53.740926 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:53.740930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:53.742547 (+ 1617us) server_negotiation.cc:407] Connection header received
0923 18:01:53.743880 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:53.743885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:53.743947 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:53.744019 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:53.745827 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:53.746801 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:53.748744 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:53.749146 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:53.749227 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:53.756002 (+ 6775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:53.756006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:53.756009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:01:53.756038 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:53.758014 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:53.758793 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:53.758797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:53.758799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:53.758930 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:53.759489 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:53.759493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:53.759494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:53.759831 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:53.759936 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:53.760673 (+ 737us) server_negotiation.cc:299] Negotiation successful
0923 18:01:53.760831 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1094,"thread_start_us":69,"threads_started":1}
I20260923 18:01:53.763689 27378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:53.739190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:45355 (local address 127.15.139.129:34779)
0923 18:01:53.742317 (+ 3127us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:53.742390 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:53.743597 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:53.744510 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:53.744520 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:53.744903 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:53.745639 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:53.745665 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:53.746980 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:53.746984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:53.748593 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:53.748607 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:53.749448 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:53.749452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:53.749669 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:53.753774 (+ 4105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:53.753829 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:53.755755 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:01:53.758158 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:01:53.758166 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:01:53.758186 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:01:53.758626 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:01:53.759051 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:01:53.759055 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:01:53.759058 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:01:53.759355 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:01:53.759996 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:01:53.760006 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:01:53.760370 (+ 364us) client_negotiation.cc:769] Sending connection context
0923 18:01:53.761438 (+ 1068us) client_negotiation.cc:240] Negotiation successful
0923 18:01:53.762281 (+ 843us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2882,"thread_start_us":142,"threads_started":1}
I20260923 18:01:53.765486 27376 heartbeater.cc:347] Connected to a master server at 127.15.139.190:45355
I20260923 18:01:53.765802 27376 heartbeater.cc:464] Registering TS with master...
I20260923 18:01:53.766420 27376 heartbeater.cc:511] Master 127.15.139.190:45355 requested a full tablet report, sending...
I20260923 18:01:53.768438 27160 ts_manager.cc:194] Registered new tserver with Master: c8742513d1584075bf829459d2ff94bb (127.15.139.129:42405)
I20260923 18:01:53.770198 27160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:34779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:01:53.782606 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.139.148:52691/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:45355
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:43193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 18:01:53 dist-test-slave-vr57 krb5kdc[26680](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790186513, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 18:01:53.918650 27382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:53.918999 27382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:53.919129 27382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:53.924189 27382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:53.924299 27382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:53.924444 27382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:01:53.931049 27382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:43193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.139.148:52691/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:45355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:53.932395 27382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:53.933756 27382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:53.947055 27389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:53.947330 27388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:53.947116 27391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:53.947561 27382 server_base.cc:1034] running on GCE node
I20260923 18:01:53.948189 27382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:53.949077 27382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:53.950281 27382 hybrid_clock.cc:648] HybridClock initialized: now 1790186513950256 us; error 39 us; skew 500 ppm
Sep 23 18:01:53 dist-test-slave-vr57 krb5kdc[26680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186513, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:01:53.954123 27382 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:01:53.955240 27382 webserver.cc:460] Webserver started at http://127.15.139.130:46795/ using document root <none> and password file <none>
I20260923 18:01:53.955981 27382 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:53.956055 27382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:53.956290 27382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 18:01:53 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186513, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:01:53.974292 27382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "f8c8fd8423984467b031fe91c68657ab"
format_stamp: "Formatted at 2026-09-23 18:01:53 on dist-test-slave-vr57"
server_key: "1cc44385e0e857f243c1fea10a3b95e9"
server_key_iv: "952271bc12fc2f7875a29b4c1f0a63b8"
server_key_version: "kuduclusterkey@0"
I20260923 18:01:53.976032 27382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "f8c8fd8423984467b031fe91c68657ab"
format_stamp: "Formatted at 2026-09-23 18:01:53 on dist-test-slave-vr57"
server_key: "1cc44385e0e857f243c1fea10a3b95e9"
server_key_iv: "952271bc12fc2f7875a29b4c1f0a63b8"
server_key_version: "kuduclusterkey@0"
I20260923 18:01:53.984653 27382 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
I20260923 18:01:54.006467 27398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:54.008222 27382 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260923 18:01:54.008445 27382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "f8c8fd8423984467b031fe91c68657ab"
format_stamp: "Formatted at 2026-09-23 18:01:53 on dist-test-slave-vr57"
server_key: "1cc44385e0e857f243c1fea10a3b95e9"
server_key_iv: "952271bc12fc2f7875a29b4c1f0a63b8"
server_key_version: "kuduclusterkey@0"
I20260923 18:01:54.008688 27382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:54.023995 27382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:54.027182 27382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:54.027397 27382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:54.028119 27382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:01:54.029037 27382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:01:54.029109 27382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:54.029167 27382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:01:54.029196 27382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:54.041584 27382 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:35503
I20260923 18:01:54.043000 27382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:01:54.044199 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 27382
I20260923 18:01:54.044348 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:01:54.044746 15918 external_mini_cluster.cc:1428] Setting key 36ee69afcac27dd869ebd48b2011bfc3
I20260923 18:01:54.054221 27511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:35503 every 8 connection(s)
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
Sep 23 18:01:54 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186513, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:01:54.075341 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.050564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:37967 (local address 127.15.139.190:45355)
0923 18:01:54.050715 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:54.050720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:54.052280 (+ 1560us) server_negotiation.cc:407] Connection header received
0923 18:01:54.053780 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:54.053783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:54.053838 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:54.053920 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:54.057071 (+ 3151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.058029 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.060491 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.060872 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.061008 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.070035 (+ 9027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:54.070041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:54.070044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:01:54.070095 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:54.072259 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:54.073123 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:54.073127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:54.073129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:54.073235 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:54.073758 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:54.073761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:54.073764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:54.074031 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:54.074144 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:54.074969 (+ 825us) server_negotiation.cc:299] Negotiation successful
0923 18:01:54.075142 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 18:01:54.077400 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.051218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:45355 (local address 127.15.139.130:37967)
0923 18:01:54.052114 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:54.052176 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:54.053547 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:54.055680 (+ 2133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:54.055689 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:54.056085 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:54.056861 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.056884 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.058608 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.058613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.060301 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.060314 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.061274 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.061289 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.066671 (+ 5382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.067652 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:54.067681 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:54.069700 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:01:54.072474 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:01:54.072482 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:01:54.072505 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:01:54.072953 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:01:54.073377 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:01:54.073382 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:01:54.073385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:01:54.073619 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:01:54.074226 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:01:54.074234 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:01:54.074630 (+ 396us) client_negotiation.cc:769] Sending connection context
0923 18:01:54.075666 (+ 1036us) client_negotiation.cc:240] Negotiation successful
0923 18:01:54.076091 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":202,"threads_started":1}
I20260923 18:01:54.078923 27512 heartbeater.cc:347] Connected to a master server at 127.15.139.190:45355
I20260923 18:01:54.079213 27512 heartbeater.cc:464] Registering TS with master...
I20260923 18:01:54.079936 27512 heartbeater.cc:511] Master 127.15.139.190:45355 requested a full tablet report, sending...
I20260923 18:01:54.081175 27160 ts_manager.cc:194] Registered new tserver with Master: f8c8fd8423984467b031fe91c68657ab (127.15.139.130:35503)
I20260923 18:01:54.081940 27160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:37967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:01:54.102515 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.139.148:52691/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:45355
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:43193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:01:54.271757 27517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:01:54.272254 27517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:01:54.272388 27517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:01:54.278757 27517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:01:54.278903 27517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:01:54.279033 27517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:01:54.286947 27517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:43193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.139.148:52691/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:45355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:01:54.288569 27517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:01:54.289925 27517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:01:54.303248 27525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:01:54.303314 27522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:54.303670 27517 server_base.cc:1034] running on GCE node
W20260923 18:01:54.304253 27523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:01:54.304793 27517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:01:54.305577 27517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:01:54.306824 27517 hybrid_clock.cc:648] HybridClock initialized: now 1790186514306760 us; error 80 us; skew 500 ppm
Sep 23 18:01:54 dist-test-slave-vr57 krb5kdc[26680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186514, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:01:54.310354 27517 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:01:54.311476 27517 webserver.cc:460] Webserver started at http://127.15.139.131:32885/ using document root <none> and password file <none>
I20260923 18:01:54.312188 27517 fs_manager.cc:359] Metadata directory not provided
I20260923 18:01:54.312261 27517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:01:54.312503 27517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 18:01:54 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186514, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
Sep 23 18:01:54 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186513, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:01:54.327317 27517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "98b2c5e942e0407fbbefcf94450c9724"
format_stamp: "Formatted at 2026-09-23 18:01:54 on dist-test-slave-vr57"
server_key: "225b98eebcd68bdcd29504cb44936557"
server_key_iv: "cd656f64ba9f4e2d317dd39c5308b006"
server_key_version: "kuduclusterkey@0"
I20260923 18:01:54.328999 27517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "98b2c5e942e0407fbbefcf94450c9724"
format_stamp: "Formatted at 2026-09-23 18:01:54 on dist-test-slave-vr57"
server_key: "225b98eebcd68bdcd29504cb44936557"
server_key_iv: "cd656f64ba9f4e2d317dd39c5308b006"
server_key_version: "kuduclusterkey@0"
I20260923 18:01:54.335045 27517 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260923 18:01:54.357519 27532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:54.359091 27517 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:01:54.359227 27517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "98b2c5e942e0407fbbefcf94450c9724"
format_stamp: "Formatted at 2026-09-23 18:01:54 on dist-test-slave-vr57"
server_key: "225b98eebcd68bdcd29504cb44936557"
server_key_iv: "cd656f64ba9f4e2d317dd39c5308b006"
server_key_version: "kuduclusterkey@0"
I20260923 18:01:54.359364 27517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:01:54.376758 27517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:01:54.380290 27517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:01:54.380477 27517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:01:54.381081 27517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:01:54.382063 27517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:01:54.382123 27517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:54.382189 27517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:01:54.382213 27517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:01:54.395466 27517 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:34127
I20260923 18:01:54.396654 27517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:01:54.397367 27646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:34127 every 8 connection(s)
I20260923 18:01:54.397776 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 27517
I20260923 18:01:54.397902 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:01:54.398211 15918 external_mini_cluster.cc:1428] Setting key 0871b2c496fca1f6f8bf2ee16eb94f7d
Sep 23 18:01:54 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186514, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:01:54.418660 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.401587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:60831 (local address 127.15.139.190:45355)
0923 18:01:54.403052 (+ 1465us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:54.403057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:54.403586 (+ 529us) server_negotiation.cc:407] Connection header received
0923 18:01:54.404763 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:54.404767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:54.404818 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:54.404868 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:54.406465 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.407331 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.409031 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.409398 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.409471 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.414403 (+ 4932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:54.414407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:54.414410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:01:54.414447 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:54.416254 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:54.416898 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:54.416901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:54.416903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:54.416987 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:54.417392 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:54.417395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:54.417396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:54.417614 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:54.417693 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:54.418382 (+ 689us) server_negotiation.cc:299] Negotiation successful
0923 18:01:54.418506 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1376}
I20260923 18:01:54.420712 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.402051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:45355 (local address 127.15.139.131:60831)
0923 18:01:54.403407 (+ 1356us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:54.403462 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:54.404528 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:54.405277 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:54.405286 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:54.405628 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:01:54.406299 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.406320 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.407488 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.407491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.408892 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.408903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.409675 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.409679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.409881 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.412320 (+ 2439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:01:54.412342 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:01:54.414125 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:01:54.416406 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:01:54.416412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:01:54.416433 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:01:54.416782 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:01:54.417089 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:01:54.417091 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:01:54.417093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:01:54.417287 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:01:54.417757 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:01:54.417764 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:01:54.418092 (+ 328us) client_negotiation.cc:769] Sending connection context
0923 18:01:54.419186 (+ 1094us) client_negotiation.cc:240] Negotiation successful
0923 18:01:54.419573 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1149,"thread_start_us":156,"threads_started":1}
I20260923 18:01:54.422662 27647 heartbeater.cc:347] Connected to a master server at 127.15.139.190:45355
I20260923 18:01:54.422909 27647 heartbeater.cc:464] Registering TS with master...
I20260923 18:01:54.423483 27647 heartbeater.cc:511] Master 127.15.139.190:45355 requested a full tablet report, sending...
I20260923 18:01:54.426143 27160 ts_manager.cc:194] Registered new tserver with Master: 98b2c5e942e0407fbbefcf94450c9724 (127.15.139.131:34127)
I20260923 18:01:54.426864 27160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:60831
I20260923 18:01:54.437573 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:01:54.449594 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.440068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46900 (local address 127.15.139.190:45355)
0923 18:01:54.440197 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:54.440204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:54.440607 (+ 403us) server_negotiation.cc:407] Connection header received
0923 18:01:54.440795 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:54.440798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:54.440845 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:54.440893 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:01:54.441850 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.442636 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.443826 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.444188 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.444263 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.446109 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:01:54.446113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:01:54.446115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:01:54.446152 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:01:54.447839 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:54.448318 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:54.448322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:54.448324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:54.448404 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:01:54.448683 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:01:54.448686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:01:54.448688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:01:54.448883 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:01:54.448976 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:54.449331 (+ 355us) server_negotiation.cc:299] Negotiation successful
0923 18:01:54.449446 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:01:54.453637 27160 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:01:54.458114 27160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:01:54.486723 27661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.469655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:42405 (local address 127.0.0.1:51344)
0923 18:01:54.472311 (+ 2656us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:54.472323 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:54.472399 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:54.473241 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:54.473248 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:54.473270 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:01:54.473416 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.473422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.474844 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.474847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.479686 (+ 4839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.479697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.486153 (+ 6456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.486156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.486294 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.486333 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 18:01:54.486442 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 18:01:54.486564 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2602,"thread_start_us":84,"threads_started":1}
I20260923 18:01:54.487447 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.470676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:34127 (local address 127.0.0.1:50806)
0923 18:01:54.475837 (+ 5161us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:54.475850 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:54.475934 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:54.477532 (+ 1598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:54.477534 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:54.477546 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:01:54.477606 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.477610 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.479812 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.479815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.480999 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.481008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.487060 (+ 6052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.487064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.487163 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.487196 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 18:01:54.487261 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 18:01:54.487324 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5099,"thread_start_us":92,"threads_started":1}
I20260923 18:01:54.487936 27664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.471043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50806 (local address 127.15.139.131:34127)
0923 18:01:54.472184 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:54.472195 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:54.476025 (+ 3830us) server_negotiation.cc:407] Connection header received
0923 18:01:54.476102 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:54.476107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:54.476313 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:54.476451 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:01:54.477713 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.478881 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.481153 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.482005 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.482122 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.482266 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:54.487684 (+ 5418us) server_negotiation.cc:299] Negotiation successful
0923 18:01:54.487800 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":140,"threads_started":1}
I20260923 18:01:54.491231 27580 tablet_service.cc:1467] Processing CreateTablet for tablet 2937bc9b7fb8455c92e1872b97845479 (DEFAULT_TABLE table=test-table [id=2b84693db83b4836967dc9f7e79cf2d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:01:54.492489 27580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2937bc9b7fb8455c92e1872b97845479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:01:54.501830 27663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.468088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:35503 (local address 127.0.0.1:46098)
0923 18:01:54.471723 (+ 3635us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:54.471774 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:54.471899 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:54.477316 (+ 5417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:54.477319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:54.477336 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:01:54.477430 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.477435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.483654 (+ 6219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.483658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.484828 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.484838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.501380 (+ 16542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.501384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.501493 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.501518 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 18:01:54.501592 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 18:01:54.501677 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3453,"thread_start_us":109,"threads_started":1}
I20260923 18:01:54.502439 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.470037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51344 (local address 127.15.139.129:42405)
0923 18:01:54.472612 (+ 2575us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:54.472618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:54.472641 (+ 23us) server_negotiation.cc:407] Connection header received
0923 18:01:54.472714 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:54.472719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:54.472929 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:54.473069 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:01:54.473543 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.474712 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.484981 (+ 10269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.485846 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.485966 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.486091 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:54.502162 (+ 16071us) server_negotiation.cc:299] Negotiation successful
0923 18:01:54.502300 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2477,"thread_start_us":81,"threads_started":1}
I20260923 18:01:54.505505 27303 tablet_service.cc:1467] Processing CreateTablet for tablet 2937bc9b7fb8455c92e1872b97845479 (DEFAULT_TABLE table=test-table [id=2b84693db83b4836967dc9f7e79cf2d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:01:54.506635 27303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2937bc9b7fb8455c92e1872b97845479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:01:54.517845 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.468910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46098 (local address 127.15.139.130:35503)
0923 18:01:54.476728 (+ 7818us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:54.476738 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:54.476760 (+ 22us) server_negotiation.cc:407] Connection header received
0923 18:01:54.476855 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:54.476860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:54.477035 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:54.477184 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:01:54.482337 (+ 5153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.483522 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.500106 (+ 16584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.501022 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.501165 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.501305 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:54.517539 (+ 16234us) server_negotiation.cc:299] Negotiation successful
0923 18:01:54.517683 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7714,"thread_start_us":104,"threads_started":1}
I20260923 18:01:54.520857 27446 tablet_service.cc:1467] Processing CreateTablet for tablet 2937bc9b7fb8455c92e1872b97845479 (DEFAULT_TABLE table=test-table [id=2b84693db83b4836967dc9f7e79cf2d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:01:54.522027 27446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2937bc9b7fb8455c92e1872b97845479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:01:54.531952 27667 tablet_bootstrap.cc:492] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab: Bootstrap starting.
I20260923 18:01:54.534332 27667 tablet_bootstrap.cc:654] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab: Neither blocks nor log segments found. Creating new log.
I20260923 18:01:54.535679 27667 log.cc:824] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab: Log is configured to *not* fsync() on all Append() calls
I20260923 18:01:54.536505 27666 tablet_bootstrap.cc:492] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb: Bootstrap starting.
I20260923 18:01:54.538774 27666 tablet_bootstrap.cc:654] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb: Neither blocks nor log segments found. Creating new log.
I20260923 18:01:54.539639 27666 log.cc:824] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb: Log is configured to *not* fsync() on all Append() calls
I20260923 18:01:54.540391 27665 tablet_bootstrap.cc:492] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724: Bootstrap starting.
I20260923 18:01:54.543378 27665 tablet_bootstrap.cc:654] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724: Neither blocks nor log segments found. Creating new log.
I20260923 18:01:54.544214 27665 log.cc:824] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724: Log is configured to *not* fsync() on all Append() calls
I20260923 18:01:54.546434 27667 tablet_bootstrap.cc:492] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab: No bootstrap required, opened a new log
I20260923 18:01:54.546635 27667 ts_tablet_manager.cc:1397] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.004s
I20260923 18:01:54.550942 27666 tablet_bootstrap.cc:492] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb: No bootstrap required, opened a new log
I20260923 18:01:54.551144 27666 ts_tablet_manager.cc:1397] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20260923 18:01:54.555646 27666 raft_consensus.cc:348] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8c8fd8423984467b031fe91c68657ab" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35503 } } peers { permanent_uuid: "c8742513d1584075bf829459d2ff94bb" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42405 } } peers { permanent_uuid: "98b2c5e942e0407fbbefcf94450c9724" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 34127 } }
I20260923 18:01:54.555951 27666 raft_consensus.cc:374] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:01:54.556007 27666 raft_consensus.cc:729] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8742513d1584075bf829459d2ff94bb, State: Initialized, Role: FOLLOWER
I20260923 18:01:54.556146 27667 raft_consensus.cc:348] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8c8fd8423984467b031fe91c68657ab" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35503 } } peers { permanent_uuid: "c8742513d1584075bf829459d2ff94bb" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42405 } } peers { permanent_uuid: "98b2c5e942e0407fbbefcf94450c9724" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 34127 } }
I20260923 18:01:54.556447 27667 raft_consensus.cc:374] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:01:54.556504 27667 raft_consensus.cc:729] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8c8fd8423984467b031fe91c68657ab, State: Initialized, Role: FOLLOWER
I20260923 18:01:54.556540 27666 consensus_queue.cc:260] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8c8fd8423984467b031fe91c68657ab" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35503 } } peers { permanent_uuid: "c8742513d1584075bf829459d2ff94bb" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42405 } } peers { permanent_uuid: "98b2c5e942e0407fbbefcf94450c9724" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 34127 } }
I20260923 18:01:54.557530 27667 consensus_queue.cc:260] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8c8fd8423984467b031fe91c68657ab" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35503 } } peers { permanent_uuid: "c8742513d1584075bf829459d2ff94bb" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42405 } } peers { permanent_uuid: "98b2c5e942e0407fbbefcf94450c9724" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 34127 } }
I20260923 18:01:54.558573 27667 ts_tablet_manager.cc:1428] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260923 18:01:54.558676 27665 tablet_bootstrap.cc:492] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724: No bootstrap required, opened a new log
I20260923 18:01:54.558898 27665 ts_tablet_manager.cc:1397] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.004s
I20260923 18:01:54.558985 27512 heartbeater.cc:503] Master 127.15.139.190:45355 was elected leader, sending a full tablet report...
I20260923 18:01:54.559530 27666 ts_tablet_manager.cc:1428] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260923 18:01:54.560038 27376 heartbeater.cc:503] Master 127.15.139.190:45355 was elected leader, sending a full tablet report...
I20260923 18:01:54.563817 27665 raft_consensus.cc:348] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8c8fd8423984467b031fe91c68657ab" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35503 } } peers { permanent_uuid: "c8742513d1584075bf829459d2ff94bb" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42405 } } peers { permanent_uuid: "98b2c5e942e0407fbbefcf94450c9724" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 34127 } }
I20260923 18:01:54.564105 27665 raft_consensus.cc:374] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:01:54.564163 27665 raft_consensus.cc:729] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98b2c5e942e0407fbbefcf94450c9724, State: Initialized, Role: FOLLOWER
I20260923 18:01:54.564635 27665 consensus_queue.cc:260] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8c8fd8423984467b031fe91c68657ab" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35503 } } peers { permanent_uuid: "c8742513d1584075bf829459d2ff94bb" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42405 } } peers { permanent_uuid: "98b2c5e942e0407fbbefcf94450c9724" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 34127 } }
I20260923 18:01:54.565712 27665 ts_tablet_manager.cc:1428] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260923 18:01:54.566017 27647 heartbeater.cc:503] Master 127.15.139.190:45355 was elected leader, sending a full tablet report...
W20260923 18:01:54.651090 27648 tablet.cc:2369] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 18:01:54 dist-test-slave-vr57 krb5kdc[26680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186513, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
W20260923 18:01:54.745654 27377 tablet.cc:2369] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:01:54.800897 27513 tablet.cc:2369] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:01:54.835590 27672 raft_consensus.cc:482] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:01:54.835846 27672 raft_consensus.cc:504] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8c8fd8423984467b031fe91c68657ab" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35503 } } peers { permanent_uuid: "c8742513d1584075bf829459d2ff94bb" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42405 } } peers { permanent_uuid: "98b2c5e942e0407fbbefcf94450c9724" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 34127 } }
I20260923 18:01:54.837553 27672 leader_election.cc:288] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8742513d1584075bf829459d2ff94bb (127.15.139.129:42405), 98b2c5e942e0407fbbefcf94450c9724 (127.15.139.131:34127)
I20260923 18:01:54.843803 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.838781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:60047 (local address 127.15.139.129:42405)
0923 18:01:54.838937 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:54.838942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:54.839277 (+ 335us) server_negotiation.cc:407] Connection header received
0923 18:01:54.839473 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:54.839476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:54.839565 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:54.839638 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:01:54.840043 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.840836 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.842432 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.843036 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.843128 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.843182 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:54.843571 (+ 389us) server_negotiation.cc:299] Negotiation successful
0923 18:01:54.843666 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 18:01:54.843840 27664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.838919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:44779 (local address 127.15.139.131:34127)
0923 18:01:54.839052 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:54.839057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:54.839736 (+ 679us) server_negotiation.cc:407] Connection header received
0923 18:01:54.839999 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:54.840004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:54.840060 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:54.840109 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:01:54.840472 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.841160 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.842573 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.843170 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.843256 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.843302 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:54.843651 (+ 349us) server_negotiation.cc:299] Negotiation successful
0923 18:01:54.843723 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 18:01:54.844237 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.837941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:42405 (local address 127.15.139.130:60047)
0923 18:01:54.839186 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:54.839201 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:54.839361 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:54.839777 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:54.839780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:54.839797 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:01:54.839925 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.839933 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.840971 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.840974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.842273 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.842283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.843291 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.843294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.843403 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.843430 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 18:01:54.843966 (+ 536us) client_negotiation.cc:240] Negotiation successful
0923 18:01:54.844069 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1156,"thread_start_us":72,"threads_started":1}
I20260923 18:01:54.844940 27329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2937bc9b7fb8455c92e1872b97845479" candidate_uuid: "f8c8fd8423984467b031fe91c68657ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8742513d1584075bf829459d2ff94bb" is_pre_election: true
I20260923 18:01:54.845296 27329 raft_consensus.cc:2454] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8c8fd8423984467b031fe91c68657ab in term 0.
I20260923 18:01:54.845875 27401 leader_election.cc:302] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8742513d1584075bf829459d2ff94bb, f8c8fd8423984467b031fe91c68657ab; no voters:
I20260923 18:01:54.846204 27672 raft_consensus.cc:2781] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:01:54.846288 27672 raft_consensus.cc:482] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:01:54.846338 27672 raft_consensus.cc:3037] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:01:54.847724 27672 raft_consensus.cc:504] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8c8fd8423984467b031fe91c68657ab" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35503 } } peers { permanent_uuid: "c8742513d1584075bf829459d2ff94bb" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42405 } } peers { permanent_uuid: "98b2c5e942e0407fbbefcf94450c9724" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 34127 } }
I20260923 18:01:54.848116 27672 leader_election.cc:288] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [CANDIDATE]: Term 1 election: Requested vote from peers c8742513d1584075bf829459d2ff94bb (127.15.139.129:42405), 98b2c5e942e0407fbbefcf94450c9724 (127.15.139.131:34127)
I20260923 18:01:54.848819 27329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2937bc9b7fb8455c92e1872b97845479" candidate_uuid: "f8c8fd8423984467b031fe91c68657ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8742513d1584075bf829459d2ff94bb"
I20260923 18:01:54.848950 27329 raft_consensus.cc:3037] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:01:54.850013 27329 raft_consensus.cc:2454] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8c8fd8423984467b031fe91c68657ab in term 1.
I20260923 18:01:54.850355 27401 leader_election.cc:302] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8742513d1584075bf829459d2ff94bb, f8c8fd8423984467b031fe91c68657ab; no voters:
I20260923 18:01:54.850512 27672 raft_consensus.cc:2781] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:01:54.850767 27677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.838286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:34127 (local address 127.15.139.130:44779)
0923 18:01:54.839654 (+ 1368us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:54.839668 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:01:54.839863 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:01:54.840231 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:01:54.840234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:01:54.840251 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:01:54.840357 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.840362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.841311 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.841314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.842453 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:01:54.842461 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.843383 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.843386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:01:54.843490 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.843544 (+ 54us) client_negotiation.cc:769] Sending connection context
0923 18:01:54.850265 (+ 6721us) client_negotiation.cc:240] Negotiation successful
0923 18:01:54.850360 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1299,"spinlock_wait_cycles":18688,"thread_start_us":105,"threads_started":1}
I20260923 18:01:54.850787 27672 raft_consensus.cc:686] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [term 1 LEADER]: Becoming Leader. State: Replica: f8c8fd8423984467b031fe91c68657ab, State: Running, Role: LEADER
I20260923 18:01:54.851382 27672 consensus_queue.cc:237] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8c8fd8423984467b031fe91c68657ab" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35503 } } peers { permanent_uuid: "c8742513d1584075bf829459d2ff94bb" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42405 } } peers { permanent_uuid: "98b2c5e942e0407fbbefcf94450c9724" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 34127 } }
I20260923 18:01:54.851989 27600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2937bc9b7fb8455c92e1872b97845479" candidate_uuid: "f8c8fd8423984467b031fe91c68657ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98b2c5e942e0407fbbefcf94450c9724" is_pre_election: true
I20260923 18:01:54.852286 27600 raft_consensus.cc:2454] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8c8fd8423984467b031fe91c68657ab in term 0.
I20260923 18:01:54.852612 27595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2937bc9b7fb8455c92e1872b97845479" candidate_uuid: "f8c8fd8423984467b031fe91c68657ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98b2c5e942e0407fbbefcf94450c9724"
I20260923 18:01:54.852717 27595 raft_consensus.cc:3037] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:01:54.853818 27595 raft_consensus.cc:2454] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8c8fd8423984467b031fe91c68657ab in term 1.
I20260923 18:01:54.855409 27160 catalog_manager.cc:5495] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab reported cstate change: term changed from 0 to 1, leader changed from <none> to f8c8fd8423984467b031fe91c68657ab (127.15.139.130). New cstate: current_term: 1 leader_uuid: "f8c8fd8423984467b031fe91c68657ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8c8fd8423984467b031fe91c68657ab" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8742513d1584075bf829459d2ff94bb" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98b2c5e942e0407fbbefcf94450c9724" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 34127 } health_report { overall_health: UNKNOWN } } }
I20260923 18:01:54.928185 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.923658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46108 (local address 127.15.139.130:35503)
0923 18:01:54.923783 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 18:01:54.923787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:01:54.923802 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:01:54.923942 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:01:54.923945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:01:54.924062 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:01:54.924341 (+ 279us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:01:54.924590 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.925401 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.926416 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:01:54.926762 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:01:54.926885 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:01:54.927173 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:01:54.927800 (+ 627us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:01:54.927917 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0923 18:01:54.927975 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 18:01:54.928033 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 18:01:54.934192 27329 raft_consensus.cc:1261] T 2937bc9b7fb8455c92e1872b97845479 P c8742513d1584075bf829459d2ff94bb [term 1 FOLLOWER]: Refusing update from remote peer f8c8fd8423984467b031fe91c68657ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:01:54.934192 27595 raft_consensus.cc:1261] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724 [term 1 FOLLOWER]: Refusing update from remote peer f8c8fd8423984467b031fe91c68657ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:01:54.934938 27672 consensus_queue.cc:1035] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "98b2c5e942e0407fbbefcf94450c9724" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 34127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:01:54.935164 27678 consensus_queue.cc:1035] T 2937bc9b7fb8455c92e1872b97845479 P f8c8fd8423984467b031fe91c68657ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8742513d1584075bf829459d2ff94bb" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:01:54.942855 27688 mvcc.cc:205] Tried to move back new op lower bound from 7332603965162774528 to 7332603964834467840. Current Snapshot: MvccSnapshot[applied={T|T < 7332603965162774528}]
I20260923 18:01:54.945253 27690 mvcc.cc:205] Tried to move back new op lower bound from 7332603965162774528 to 7332603964834467840. Current Snapshot: MvccSnapshot[applied={T|T < 7332603965162774528}]
I20260923 18:01:54.946384 27689 mvcc.cc:205] Tried to move back new op lower bound from 7332603965162774528 to 7332603964834467840. Current Snapshot: MvccSnapshot[applied={T|T < 7332603965162774528}]
I20260923 18:01:54.954493 27160 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:01:54.954674 27160 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:01:54.957062 27160 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4cd225de1514279a3b0bb696f5e2c69: Sending DeleteTablet for 3 replicas of tablet 2937bc9b7fb8455c92e1872b97845479
I20260923 18:01:54.957821 27446 tablet_service.cc:1514] Processing DeleteTablet for tablet 2937bc9b7fb8455c92e1872b97845479 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:01:54 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:46098
I20260923 18:01:54.957836 27303 tablet_service.cc:1514] Processing DeleteTablet for tablet 2937bc9b7fb8455c92e1872b97845479 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:01:54 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:51344
I20260923 18:01:54.958282 27580 tablet_service.cc:1514] Processing DeleteTablet for tablet 2937bc9b7fb8455c92e1872b97845479 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:01:54 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:50806
I20260923 18:01:54.958734 27699 tablet_replica.cc:330] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724: stopping tablet replica
I20260923 18:01:54.959084 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 27242
I20260923 18:01:54.959112 27699 raft_consensus.cc:2229] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:01:54.959981 27699 raft_consensus.cc:2258] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:01:54.963990 27699 ts_tablet_manager.cc:1905] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:01:54.967674 27699 ts_tablet_manager.cc:1918] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:01:54.967852 27699 log.cc:1196] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/2937bc9b7fb8455c92e1872b97845479
I20260923 18:01:54.968276 27699 ts_tablet_manager.cc:1939] T 2937bc9b7fb8455c92e1872b97845479 P 98b2c5e942e0407fbbefcf94450c9724: Deleting consensus metadata
W20260923 18:01:54.968797 27146 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:42405 (error 108)
I20260923 18:01:54.969702 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 27382
I20260923 18:01:54.971699 27146 catalog_manager.cc:4841] TS 98b2c5e942e0407fbbefcf94450c9724 (127.15.139.131:34127): tablet 2937bc9b7fb8455c92e1872b97845479 (table test-table [id=2b84693db83b4836967dc9f7e79cf2d6]) successfully deleted
I20260923 18:01:54.971807 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.971544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:42405 (local address 127.0.0.1:51348)
0923 18:01:54.971652 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:54.971723 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 18:01:54.971900 27146 catalog_manager.cc:4584] TS c8742513d1584075bf829459d2ff94bb (127.15.139.129:42405): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2937bc9b7fb8455c92e1872b97845479: Network error: Client connection negotiation failed: client connection to 127.15.139.129:42405: connect: Connection refused (error 111)
I20260923 18:01:54.977176 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 27517
I20260923 18:01:54.978240 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:01:54.977293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:35503 (local address 127.0.0.1:46120)
0923 18:01:54.978082 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0923 18:01:54.978143 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.130:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":704}
I20260923 18:01:54.993937 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 27128
I20260923 18:01:55.002545 15918 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 18:01:55.347630 15918 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 18:01:55.347798 15918 mini_ranger.cc:66] Stopping Ranger...
I20260923 18:01:55.692793 15918 mini_ranger.cc:68] Stopped Ranger
2026-09-23 18:01:55.693 UTC [26710] LOG: received smart shutdown request
2026-09-23 18:01:55.696 UTC [26710] LOG: background worker "logical replication launcher" (PID 26719) exited with exit code 1
2026-09-23 18:01:55.696 UTC [26714] LOG: shutting down
2026-09-23 18:01:55.863 UTC [26710] LOG: database system is shut down
2026-09-23T18:01:55Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48438 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:01:56 dist-test-slave-vr57 krb5kdc[27709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:01:56 dist-test-slave-vr57 krb5kdc[27709](info): set up 2 sockets
Sep 23 18:01:56 dist-test-slave-vr57 krb5kdc[27709](info): commencing operation
krb5kdc: starting...
W20260923 18:01:57.044389 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:01:57 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:01:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:01:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.139.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.139.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/rangeradmin_127.15.139.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/rangeradmin_127.15.139.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.139.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.139.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/rangerlookup_127.15.139.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/rangerlookup_127.15.139.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.139.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.139.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/HTTP_127.15.139.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/HTTP_127.15.139.148@KRBTEST.COM.keytab.
I20260923 18:01:57.217902 15918 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwW7RpU/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/postgres -l logfile start
2026-09-23 18:01:59.761 UTC [27739] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 18:01:59.761 UTC [27739] LOG: listening on IPv4 address "127.15.139.148", port 57155
2026-09-23 18:01:59.765 UTC [27739] LOG: listening on Unix socket "/tmp/.s.PGSQL.57155"
2026-09-23 18:01:59.800 UTC [27742] LOG: database system was shut down at 2026-09-23 18:01:58 UTC
2026-09-23 18:01:59.803 UTC [27739] LOG: database system is ready to accept connections
I20260923 18:02:00.771915 15918 mini_postgres.cc:96] Postgres bound to 57155
2026-09-23 18:02:00.778 UTC [27750] FATAL: database "slave" does not exist
127.15.139.148:57155 - accepting connections
I20260923 18:02:00.778918 15918 mini_ranger.cc:161] Starting Ranger...
I20260923 18:02:00.793975 15918 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 18:02:01.290807 15918 mini_ranger.cc:87] Created ranger Postgres database
I20260923 18:02:01.290902 15918 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-admin
2026-09-23 18:02:01,578 [I] DB FLAVOR :POSTGRES
2026-09-23 18:02:01,578 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 18:02:01,578 [I] Checking connection..
2026-09-23 18:02:01,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 18:02:01,834 [I] Checking connection passed.
2026-09-23 18:02:01,834 [I] --------- Verifying version history table ---------
2026-09-23 18:02:01,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 18:02:02,073 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 18:02:02,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 18:02:02,342 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 18:02:02,342 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 18:02:02,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 18:02:02.587 UTC [27814] WARNING: there is no transaction in progress
2026-09-23 18:02:02,593 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 18:02:02,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 18:02:02,834 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 18:02:02,834 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 18:02:02,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 18:02:03,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 18:02:03,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vr57.c.gcp-upstream.internal','N') ;"
2026-09-23 18:02:03,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 18:02:03,888 [I] Table x_portal_user does not exist in database ranger
2026-09-23 18:02:03,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 18:02:04,159 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 18:02:04,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 18:02:04,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 18:02:04,653 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 18:02:04,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 18:02:05.704 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.721 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.738 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.754 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.773 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.791 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.850 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.857 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.872 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.884 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.895 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.910 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.920 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.927 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.934 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:05.940 UTC [27922] WARNING: there is no transaction in progress
2026-09-23 18:02:06,594 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 18:02:06,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 18:02:06,820 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 18:02:06,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 18:02:07,083 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 18:02:07,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 18:02:07,334 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 18:02:07,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 18:02:07,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 18:02:07,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vr57.c.gcp-upstream.internal';"
2026-09-23 18:02:08,093 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 18:02:08,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 18:02:08,347 [I] DB_PATCHES have already been applied
I20260923 18:02:08.353084 15918 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwW7RpU/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 18:02:08.353155 15918 mini_ranger.cc:193] Using host: 127.15.139.148
I20260923 18:02:08.356891 15918 mini_ranger.cc:227] Ranger admin URL: http://127.15.139.148:46551
Sep 23, 2026 6:02:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 6:02:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 6:02:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 6:02:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 6:02:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 6:02:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 6:02:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 18:02:09.481396 15918 mini_ranger.cc:160] Time spent starting Ranger: real 8.702s user 0.005s sys 0.005s
Sep 23, 2026 6:02:30 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 6:02:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 6:02:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 6:02:31 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 6:02:31 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 18:02:32.572201 15918 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.139.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.139.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/rangerkms_127.15.139.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.139.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/rangerkms_127.15.139.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.139.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.139.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.139.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/HTTP_127.15.139.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.139.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/HTTP_127.15.139.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 18:02:32 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186552, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 18:02:32.664577 15918 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 18:02:32.675220 15918 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 18:02:33.534118 15918 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 18:02:33.534214 15918 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-kms
I20260923 18:02:33.534235 15918 mini_ranger_kms.cc:227] Using postgres at 127.15.139.148:57155
2026-09-23 18:02:33,704 [I] DB FLAVOR :POSTGRES
2026-09-23 18:02:33,704 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 18:02:33,705 [I] Checking connection
2026-09-23 18:02:33,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 18:02:33,943 [I] connection success
2026-09-23 18:02:33,943 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 18:02:33,943 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 18:02:33,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 18:02:34,200 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 18:02:34,201 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 18:02:34,201 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 18:02:34,201 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.139.148:57155/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 18:02:34,466 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 18:02:34.936950 15918 mini_ranger_kms.cc:310] Created kms service
I20260923 18:02:35.160859 15918 mini_ranger_kms.cc:326] Created kudu user
I20260923 18:02:35.211790 15918 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 18:02:35 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186552, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:02:35.664535 15918 mini_ranger_kms.cc:384] Added ranger policy
I20260923 18:02:35.664695 15918 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwW7RpU/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwW7RpU/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 18:02:35.664721 15918 mini_ranger_kms.cc:240] Using host: 127.15.139.148
I20260923 18:02:35.668548 15918 mini_ranger_kms.cc:276] Ranger KMS PID: 28103
I20260923 18:02:35.668634 15918 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.139.148:33791
Sep 23, 2026 6:02:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 6:02:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 6:02:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 6:02:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 6:02:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 6:02:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 18:02:37.964319 15918 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.300s user 0.012s sys 0.143s
I20260923 18:02:37.964459 15918 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 18:02:37.964542 15918 mini_ranger_kms.cc:401] 127.15.139.148:33791/kms/v1/keys
Sep 23 18:02:38 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790186558, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.139.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 18:02:38 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186558, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.139.148@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
Sep 23 18:02:38 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186558, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.139.148@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
Sep 23 18:02:41 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:02:41.086838 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40443
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:40443
--ranger_config_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.139.148:33791/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:41.217679 28153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:41.217978 28153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:41.218017 28153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:41.221933 28153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:02:41.221997 28153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:41.222016 28153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:41.222033 28153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:02:41.222047 28153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:02:41.228120 28153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.139.148:33791/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:40443
--ranger_config_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:41.229161 28153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:41.230605 28153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 18:02:41.240775 28153 server_base.cc:1034] running on GCE node
W20260923 18:02:41.240962 28161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:41.241019 28158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:41.240993 28159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:41.241730 28153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:41.242956 28153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:41.244141 28153 hybrid_clock.cc:648] HybridClock initialized: now 1790186561244118 us; error 36 us; skew 500 ppm
Sep 23 18:02:41 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186561, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:41.247408 28153 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:02:41.248503 28153 webserver.cc:460] Webserver started at http://127.15.139.190:34715/ using document root <none> and password file <none>
I20260923 18:02:41.249111 28153 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:41.249174 28153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:41.249460 28153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 18:02:41 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186561, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:02:41.307993 28153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "9174b54cc00a4ee68764cd38bbe2a799"
format_stamp: "Formatted at 2026-09-23 18:02:41 on dist-test-slave-vr57"
server_key: "6ba028c0cbf28e20eaafae13fb377023"
server_key_iv: "ebaff061d52fbbb32258d8abc17c815f"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:41.308701 28153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "9174b54cc00a4ee68764cd38bbe2a799"
format_stamp: "Formatted at 2026-09-23 18:02:41 on dist-test-slave-vr57"
server_key: "6ba028c0cbf28e20eaafae13fb377023"
server_key_iv: "ebaff061d52fbbb32258d8abc17c815f"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:41.313113 28153 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:02:41.332785 28168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:41.334353 28153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:02:41.334508 28153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "9174b54cc00a4ee68764cd38bbe2a799"
format_stamp: "Formatted at 2026-09-23 18:02:41 on dist-test-slave-vr57"
server_key: "6ba028c0cbf28e20eaafae13fb377023"
server_key_iv: "ebaff061d52fbbb32258d8abc17c815f"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:41.334671 28153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:41.355413 28153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:41.358685 28153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:41.358899 28153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:41.367352 28153 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:40443
I20260923 18:02:41.367357 28220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:40443 every 8 connection(s)
I20260923 18:02:41.368474 28153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:02:41.371941 28221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:41.376091 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 28153
I20260923 18:02:41.376230 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:02:41.376525 15918 external_mini_cluster.cc:1428] Setting key 418a02eae1d8a40ac0858439d11d5a09
I20260923 18:02:41.379323 28221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799: Bootstrap starting.
I20260923 18:02:41.382153 28221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:41.383054 28221 log.cc:824] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:41.385077 28221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799: No bootstrap required, opened a new log
Sep 23 18:02:41 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:41.389688 28221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9174b54cc00a4ee68764cd38bbe2a799" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40443 } }
I20260923 18:02:41.389981 28221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:41.390069 28221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9174b54cc00a4ee68764cd38bbe2a799, State: Initialized, Role: FOLLOWER
I20260923 18:02:41.390669 28221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9174b54cc00a4ee68764cd38bbe2a799" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40443 } }
I20260923 18:02:41.390873 28221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:02:41.390971 28221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:02:41.391069 28221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:41.391940 28224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:41.378026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52642 (local address 127.15.139.190:40443)
0923 18:02:41.378442 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:41.378460 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:41.378496 (+ 36us) server_negotiation.cc:407] Connection header received
0923 18:02:41.379110 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:41.379148 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:41.379597 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:41.379941 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:41.380768 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:41.381752 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:41.382700 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:41.383214 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:41.383396 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:41.386484 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:41.386491 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:41.386509 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:41.386539 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:41.388622 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:41.389137 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:41.389144 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:41.389151 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:41.389283 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:41.389590 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:41.389593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:41.389594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:41.390131 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:41.390357 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:41.390601 (+ 244us) server_negotiation.cc:299] Negotiation successful
0923 18:02:41.390890 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":140,"threads_started":1}
I20260923 18:02:41.392253 28221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9174b54cc00a4ee68764cd38bbe2a799" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40443 } }
I20260923 18:02:41.392647 28221 leader_election.cc:302] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9174b54cc00a4ee68764cd38bbe2a799; no voters:
I20260923 18:02:41.392995 28221 leader_election.cc:288] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:02:41.393153 28226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:02:41.393414 28226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 1 LEADER]: Becoming Leader. State: Replica: 9174b54cc00a4ee68764cd38bbe2a799, State: Running, Role: LEADER
I20260923 18:02:41.393769 28226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9174b54cc00a4ee68764cd38bbe2a799" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40443 } }
I20260923 18:02:41.394296 28221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:02:41.395006 28227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9174b54cc00a4ee68764cd38bbe2a799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9174b54cc00a4ee68764cd38bbe2a799" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40443 } } }
I20260923 18:02:41.395010 28226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9174b54cc00a4ee68764cd38bbe2a799. Latest consensus state: current_term: 1 leader_uuid: "9174b54cc00a4ee68764cd38bbe2a799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9174b54cc00a4ee68764cd38bbe2a799" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40443 } } }
I20260923 18:02:41.395339 28226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:41.395581 28227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:41.395742 28235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:02:41.398910 28235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:02:41.399255 28221 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 18:02:41.411126 28235 catalog_manager.cc:1347] Generated new cluster ID: 752ffc329ccd42b884392f9ad6cc93a9
I20260923 18:02:41.411319 28235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:02:41.426930 28235 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:02:41.428442 28235 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:02:41.440326 28235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799: Generated new TSK 0
I20260923 18:02:41.441751 28235 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:02:42.163736 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.139.148:33791/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40443
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:42.286329 28266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:42.286590 28266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:42.286633 28266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:42.291287 28266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:42.291379 28266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:42.291481 28266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:02:42.297369 28266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.139.148:33791/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:42.298503 28266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:42.299615 28266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:42.311043 28272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:42.311046 28271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:42.311483 28274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:42.312000 28266 server_base.cc:1034] running on GCE node
I20260923 18:02:42.312490 28266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:42.313220 28266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:42.314419 28266 hybrid_clock.cc:648] HybridClock initialized: now 1790186562314385 us; error 53 us; skew 500 ppm
Sep 23 18:02:42 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186562, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:42.323412 28266 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:02:42.324494 28266 webserver.cc:460] Webserver started at http://127.15.139.129:33807/ using document root <none> and password file <none>
I20260923 18:02:42.325244 28266 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:42.325340 28266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:42.325665 28266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 18:02:42 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186562, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:02:42.344455 28266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "79abdac46b3141a9a2a7bb69e166d627"
format_stamp: "Formatted at 2026-09-23 18:02:42 on dist-test-slave-vr57"
server_key: "f107ea0efde1c683841d43010a85bc89"
server_key_iv: "53bd3ad688a72fdb76794919c62cfd20"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:42.345237 28266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "79abdac46b3141a9a2a7bb69e166d627"
format_stamp: "Formatted at 2026-09-23 18:02:42 on dist-test-slave-vr57"
server_key: "f107ea0efde1c683841d43010a85bc89"
server_key_iv: "53bd3ad688a72fdb76794919c62cfd20"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:42.350425 28266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260923 18:02:42.373456 28283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:42.375233 28266 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.001s
I20260923 18:02:42.375425 28266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "79abdac46b3141a9a2a7bb69e166d627"
format_stamp: "Formatted at 2026-09-23 18:02:42 on dist-test-slave-vr57"
server_key: "f107ea0efde1c683841d43010a85bc89"
server_key_iv: "53bd3ad688a72fdb76794919c62cfd20"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:42.375653 28266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:42.407162 28266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:42.411119 28266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:42.411341 28266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:42.412068 28266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:42.413117 28266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:02:42.413249 28266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:42.413385 28266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:02:42.413465 28266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:42.427039 28266 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:44923
I20260923 18:02:42.427062 28396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:44923 every 8 connection(s)
I20260923 18:02:42.428308 28266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:02:42.435178 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 28266
I20260923 18:02:42.435392 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:02:42.435760 15918 external_mini_cluster.cc:1428] Setting key db2dc024d7cbeca9ae37692b20af96a3
Sep 23 18:02:42 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186562, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:42.444598 28399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:42.430271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:35343 (local address 127.15.139.190:40443)
0923 18:02:42.430592 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:42.430601 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:42.431426 (+ 825us) server_negotiation.cc:407] Connection header received
0923 18:02:42.432491 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:42.432494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:42.432554 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:42.432671 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:42.433761 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:42.434800 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:42.436505 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:42.436825 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:42.436920 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:42.440695 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:42.440701 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:42.440704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:42.440732 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:42.442356 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:42.442908 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:42.442911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:42.442913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:42.442999 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:42.443363 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:42.443366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:42.443367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:42.443634 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:42.443738 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:42.444279 (+ 541us) server_negotiation.cc:299] Negotiation successful
0923 18:02:42.444410 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":106,"threads_started":1}
I20260923 18:02:42.445540 28400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:42.430812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40443 (local address 127.15.139.129:35343)
0923 18:02:42.431268 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:42.431321 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:42.432252 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:42.432867 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:42.432874 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:42.433138 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:42.433612 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:42.433625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:42.434947 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:42.434951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:42.436376 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:42.436388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:42.437083 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:42.437088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:42.437538 (+ 450us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:42.438240 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:42.438267 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:42.440185 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:42.442486 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:42.442492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:42.442509 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:42.442796 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:42.443101 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:42.443104 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:42.443106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:42.443260 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:42.443757 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:42.443764 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:42.444020 (+ 256us) client_negotiation.cc:769] Sending connection context
0923 18:02:42.444229 (+ 209us) client_negotiation.cc:240] Negotiation successful
0923 18:02:42.444563 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":129,"threads_started":1}
I20260923 18:02:42.446996 28397 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40443
I20260923 18:02:42.447247 28397 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:42.447898 28397 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
I20260923 18:02:42.449697 28185 ts_manager.cc:194] Registered new tserver with Master: 79abdac46b3141a9a2a7bb69e166d627 (127.15.139.129:44923)
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
I20260923 18:02:42.451232 28185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:35343
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:02:42.478653 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.139.148:33791/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40443
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 18:02:42 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790186562, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 18:02:42.610958 28405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:42.611238 28405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:42.611311 28405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:42.615768 28405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:42.615839 28405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:42.615901 28405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:02:42.622311 28405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.139.148:33791/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:42.623610 28405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:42.624848 28405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:42.634770 28411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:42.635169 28414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:42.635308 28405 server_base.cc:1034] running on GCE node
W20260923 18:02:42.635370 28412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:42.635810 28405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:42.636536 28405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:42.637719 28405 hybrid_clock.cc:648] HybridClock initialized: now 1790186562637691 us; error 51 us; skew 500 ppm
Sep 23 18:02:42 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186562, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:42.641182 28405 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:02:42.642159 28405 webserver.cc:460] Webserver started at http://127.15.139.130:38535/ using document root <none> and password file <none>
I20260923 18:02:42.642792 28405 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:42.642882 28405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:42.643126 28405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 18:02:42 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186562, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:02:42.662341 28405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "5797e27fed92452db950af253d3a1e05"
format_stamp: "Formatted at 2026-09-23 18:02:42 on dist-test-slave-vr57"
server_key: "fa9c6858af48cd1a2798c03015c5d7ca"
server_key_iv: "c9f72d2672b7af72591f0499d899c8ef"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:42.663134 28405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "5797e27fed92452db950af253d3a1e05"
format_stamp: "Formatted at 2026-09-23 18:02:42 on dist-test-slave-vr57"
server_key: "fa9c6858af48cd1a2798c03015c5d7ca"
server_key_iv: "c9f72d2672b7af72591f0499d899c8ef"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:42.668766 28405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260923 18:02:42.691023 28421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:42.692811 28405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:02:42.692950 28405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "5797e27fed92452db950af253d3a1e05"
format_stamp: "Formatted at 2026-09-23 18:02:42 on dist-test-slave-vr57"
server_key: "fa9c6858af48cd1a2798c03015c5d7ca"
server_key_iv: "c9f72d2672b7af72591f0499d899c8ef"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:42.693086 28405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:42.731343 28405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:42.735440 28405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:42.735749 28405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:42.736578 28405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:42.737726 28405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:02:42.737844 28405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:42.737990 28405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:02:42.738071 28405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:42.748760 28405 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:39811
I20260923 18:02:42.750023 28405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:02:42.751130 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 28405
I20260923 18:02:42.751264 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:02:42.751609 15918 external_mini_cluster.cc:1428] Setting key d0b642728562e7300db2ea1a3feffde0
I20260923 18:02:42.761241 28534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:39811 every 8 connection(s)
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
Sep 23 18:02:42 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186562, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
I20260923 18:02:42.794435 28399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:42.766121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:52341 (local address 127.15.139.190:40443)
0923 18:02:42.766281 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:42.766286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:42.772185 (+ 5899us) server_negotiation.cc:407] Connection header received
0923 18:02:42.773657 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:42.773662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:42.773734 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:42.773801 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:42.775819 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:42.776698 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:42.778595 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:42.778981 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:42.779080 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:42.789469 (+ 10389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:42.789474 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:42.789477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:42.789518 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:42.791539 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:42.792388 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:42.792392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:42.792394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:42.792490 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:42.792986 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:42.792990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:42.792991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:42.793245 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:42.793332 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:42.794110 (+ 778us) server_negotiation.cc:299] Negotiation successful
0923 18:02:42.794256 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 18:02:42.796535 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:42.764881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40443 (local address 127.15.139.130:52341)
0923 18:02:42.771932 (+ 7051us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:42.772031 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:42.773331 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:42.774339 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:42.774351 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:42.774777 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:42.775614 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:42.775638 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:42.776900 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:42.776904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:42.778424 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:42.778437 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:42.779316 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:42.779320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:42.779565 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:42.780360 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:42.780385 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:42.789145 (+ 8760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:42.791743 (+ 2598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:42.791752 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:42.791775 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:42.792246 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:42.792616 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:42.792619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:42.792622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:42.792844 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:42.793405 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:42.793413 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:42.793794 (+ 381us) client_negotiation.cc:769] Sending connection context
0923 18:02:42.794947 (+ 1153us) client_negotiation.cc:240] Negotiation successful
0923 18:02:42.795334 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6715,"thread_start_us":185,"threads_started":1}
I20260923 18:02:42.798400 28535 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40443
I20260923 18:02:42.798691 28535 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:42.799355 28535 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
I20260923 18:02:42.800843 28185 ts_manager.cc:194] Registered new tserver with Master: 5797e27fed92452db950af253d3a1e05 (127.15.139.130:39811)
I20260923 18:02:42.801666 28185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:52341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:02:42.835434 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.139.148:33791/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40443
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:42.996446 28540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:42.996927 28540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:42.997056 28540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:43.003547 28540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:43.003715 28540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:43.003883 28540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:02:43.012372 28540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.139.148:33791/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:40443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:43.014163 28540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:43.015717 28540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:43.030186 28546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:43.030788 28545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:43.031708 28548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:43.032107 28540 server_base.cc:1034] running on GCE node
I20260923 18:02:43.032678 28540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:43.033483 28540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:43.036742 28540 hybrid_clock.cc:648] HybridClock initialized: now 1790186563036665 us; error 93 us; skew 500 ppm
Sep 23 18:02:43 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186563, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:43.041193 28540 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:02:43.042440 28540 webserver.cc:460] Webserver started at http://127.15.139.131:46527/ using document root <none> and password file <none>
I20260923 18:02:43.043255 28540 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:43.043408 28540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:43.043751 28540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 18:02:43 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186563, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:02:43.061192 28540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "b2a64900cc244da8b36d014924965f57"
format_stamp: "Formatted at 2026-09-23 18:02:43 on dist-test-slave-vr57"
server_key: "f0ff9b30db32162d168af58e1070042e"
server_key_iv: "598a99e7cd46e55b8ff2ece3373ca937"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:43.073158 28540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "b2a64900cc244da8b36d014924965f57"
format_stamp: "Formatted at 2026-09-23 18:02:43 on dist-test-slave-vr57"
server_key: "f0ff9b30db32162d168af58e1070042e"
server_key_iv: "598a99e7cd46e55b8ff2ece3373ca937"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:43.078554 28540 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260923 18:02:43.100520 28555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:43.102388 28540 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260923 18:02:43.102519 28540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "b2a64900cc244da8b36d014924965f57"
format_stamp: "Formatted at 2026-09-23 18:02:43 on dist-test-slave-vr57"
server_key: "f0ff9b30db32162d168af58e1070042e"
server_key_iv: "598a99e7cd46e55b8ff2ece3373ca937"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:43.102650 28540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:43.117592 28540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:43.121563 28540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:43.121831 28540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:43.122548 28540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:43.123663 28540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:02:43.123772 28540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:43.123867 28540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:02:43.123929 28540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:43.134227 28540 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:46687
I20260923 18:02:43.139997 28540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:02:43.138684 28668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:46687 every 8 connection(s)
I20260923 18:02:43.141347 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 28540
I20260923 18:02:43.141459 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:02:43.141753 15918 external_mini_cluster.cc:1428] Setting key dad5b11af1183c073ca0dfa43a5a2e04
Sep 23 18:02:43 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186563, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
Sep 23 18:02:43 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186562, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:02:43.170709 28399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:43.147574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:59435 (local address 127.15.139.190:40443)
0923 18:02:43.147696 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:43.147700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:43.148151 (+ 451us) server_negotiation.cc:407] Connection header received
0923 18:02:43.149640 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:43.149643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:43.149698 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:43.149757 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:43.151481 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:43.152272 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:43.155986 (+ 3714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:43.156328 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:43.156400 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:43.159798 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:43.159803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:43.159806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:43.159835 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:43.165125 (+ 5290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:43.166714 (+ 1589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:43.166719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:43.166721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:43.166877 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:43.168345 (+ 1468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:43.168350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:43.168352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:43.168661 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:43.168758 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:43.170354 (+ 1596us) server_negotiation.cc:299] Negotiation successful
0923 18:02:43.170525 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 18:02:43.173199 28671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:43.146398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40443 (local address 127.15.139.131:59435)
0923 18:02:43.147992 (+ 1594us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:43.148051 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:43.149384 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:43.150191 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:43.150200 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:43.150558 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:43.151313 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:43.151332 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:43.152703 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:43.152706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:43.154115 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:43.154126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:43.156642 (+ 2516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:43.156646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:43.156856 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:43.157605 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:43.157626 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:43.159587 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:43.165426 (+ 5839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:43.165436 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:43.165457 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:43.166457 (+ 1000us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:43.167112 (+ 655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:43.167232 (+ 120us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:43.167234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:43.168038 (+ 804us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:43.169595 (+ 1557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:43.169603 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:43.169993 (+ 390us) client_negotiation.cc:769] Sending connection context
0923 18:02:43.171675 (+ 1682us) client_negotiation.cc:240] Negotiation successful
0923 18:02:43.172087 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1378,"thread_start_us":178,"threads_started":1}
I20260923 18:02:43.174650 28669 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40443
I20260923 18:02:43.174883 28669 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:43.175434 28669 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
I20260923 18:02:43.176777 28185 ts_manager.cc:194] Registered new tserver with Master: b2a64900cc244da8b36d014924965f57 (127.15.139.131:46687)
I20260923 18:02:43.177476 28185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:59435
I20260923 18:02:43.188241 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:02:43.189077 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 28266
I20260923 18:02:43.206384 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 28405
I20260923 18:02:43.214056 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 28540
I20260923 18:02:43.223455 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 28153
I20260923 18:02:43.232218 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40443
--webserver_interface=127.15.139.190
--webserver_port=34715
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:40443
--ranger_config_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.139.148:33791/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.139.148:33791/kms with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:43.366236 28672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:43.366535 28672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:43.366616 28672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:43.370603 28672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:02:43.370700 28672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:43.370738 28672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:43.370775 28672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:02:43.370859 28672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:02:43.376356 28672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.139.148:33791/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:40443
--ranger_config_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=34715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:43.377453 28672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:43.378492 28672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:43.387287 28680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:43.387256 28678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:43.387348 28677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:43.387639 28672 server_base.cc:1034] running on GCE node
I20260923 18:02:43.388216 28672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:43.389334 28672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:43.390527 28672 hybrid_clock.cc:648] HybridClock initialized: now 1790186563390509 us; error 28 us; skew 500 ppm
Sep 23 18:02:43 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186563, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:43.393478 28672 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:02:43.394394 28672 webserver.cc:460] Webserver started at http://127.15.139.190:34715/ using document root <none> and password file <none>
I20260923 18:02:43.394906 28672 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:43.394973 28672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 18:02:43 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186563, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:02:43.468063 28672 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260923 18:02:43.470504 28688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:43.472057 28672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:02:43.472219 28672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "9174b54cc00a4ee68764cd38bbe2a799"
format_stamp: "Formatted at 2026-09-23 18:02:41 on dist-test-slave-vr57"
server_key: "6ba028c0cbf28e20eaafae13fb377023"
server_key_iv: "ebaff061d52fbbb32258d8abc17c815f"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:43.472852 28672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:43.484905 28672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:43.487967 28672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:43.488428 28672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:43.496112 28672 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:40443
I20260923 18:02:43.496119 28740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:40443 every 8 connection(s)
I20260923 18:02:43.497200 28672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:02:43.501554 28741 sys_catalog.cc:263] Verifying existing consensus state
I20260923 18:02:43.502626 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 28672
I20260923 18:02:43.503055 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.139.148:33791/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:44923
--local_ip_for_outbound_sockets=127.15.139.129
--tserver_master_addrs=127.15.139.190:40443
--webserver_port=33807
--webserver_interface=127.15.139.129
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.139.148:33791/kms with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:02:43.503207 28741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799: Bootstrap starting.
I20260923 18:02:43.519585 28741 log.cc:824] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:43.526274 28741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 18:02:43.526695 28741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799: Bootstrap complete.
I20260923 18:02:43.530939 28741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9174b54cc00a4ee68764cd38bbe2a799" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40443 } }
I20260923 18:02:43.531340 28741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9174b54cc00a4ee68764cd38bbe2a799, State: Initialized, Role: FOLLOWER
I20260923 18:02:43.531859 28741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9174b54cc00a4ee68764cd38bbe2a799" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40443 } }
I20260923 18:02:43.531978 28741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:02:43.532065 28741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:02:43.532161 28741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 1 FOLLOWER]: Advancing to term 2
I20260923 18:02:43.534014 28741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9174b54cc00a4ee68764cd38bbe2a799" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40443 } }
I20260923 18:02:43.534350 28741 leader_election.cc:302] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9174b54cc00a4ee68764cd38bbe2a799; no voters:
I20260923 18:02:43.534677 28741 leader_election.cc:288] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 18:02:43.534768 28745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 18:02:43.535022 28745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [term 2 LEADER]: Becoming Leader. State: Replica: 9174b54cc00a4ee68764cd38bbe2a799, State: Running, Role: LEADER
I20260923 18:02:43.535369 28745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9174b54cc00a4ee68764cd38bbe2a799" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40443 } }
I20260923 18:02:43.535991 28741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:02:43.536329 28745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9174b54cc00a4ee68764cd38bbe2a799. Latest consensus state: current_term: 2 leader_uuid: "9174b54cc00a4ee68764cd38bbe2a799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9174b54cc00a4ee68764cd38bbe2a799" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40443 } } }
I20260923 18:02:43.536444 28745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:43.536872 28749 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:02:43.536832 28746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9174b54cc00a4ee68764cd38bbe2a799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9174b54cc00a4ee68764cd38bbe2a799" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40443 } } }
I20260923 18:02:43.537168 28746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799 [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:43.540184 28749 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:02:43.540688 28741 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 18:02:43.540802 28749 catalog_manager.cc:1259] Loaded cluster ID: 752ffc329ccd42b884392f9ad6cc93a9
I20260923 18:02:43.540850 28749 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:02:43.543162 28749 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:02:43.543841 28749 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799: Loaded TSK: 0
I20260923 18:02:43.545046 28749 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 18:02:43.645812 28743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:43.646205 28743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:43.646296 28743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:43.653051 28743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:43.653137 28743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:43.653231 28743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:02:43.661072 28743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.139.148:33791/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:44923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=33807
--tserver_master_addrs=127.15.139.190:40443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:43.662372 28743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:43.663700 28743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:43.676002 28781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:43.676056 28779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:43.676463 28778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:43.676717 28743 server_base.cc:1034] running on GCE node
I20260923 18:02:43.677134 28743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:43.677814 28743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:43.678964 28743 hybrid_clock.cc:648] HybridClock initialized: now 1790186563678935 us; error 44 us; skew 500 ppm
Sep 23 18:02:43 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186563, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:43.682472 28743 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:02:43.683641 28743 webserver.cc:460] Webserver started at http://127.15.139.129:33807/ using document root <none> and password file <none>
I20260923 18:02:43.684558 28743 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:43.684751 28743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 18:02:43 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186563, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:02:43.715088 28743 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260923 18:02:43.718571 28789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:43.720304 28743 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260923 18:02:43.720455 28743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "79abdac46b3141a9a2a7bb69e166d627"
format_stamp: "Formatted at 2026-09-23 18:02:42 on dist-test-slave-vr57"
server_key: "f107ea0efde1c683841d43010a85bc89"
server_key_iv: "53bd3ad688a72fdb76794919c62cfd20"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:43.721236 28743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:43.748484 28743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:43.752653 28743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:43.752894 28743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:43.753683 28743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:43.754837 28743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:02:43.754915 28743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:43.754977 28743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:02:43.755009 28743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:43.766768 28743 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:44923
I20260923 18:02:43.766942 28902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:44923 every 8 connection(s)
I20260923 18:02:43.768031 28743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:02:43.772872 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 28743
I20260923 18:02:43.773411 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.139.148:33791/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:39811
--local_ip_for_outbound_sockets=127.15.139.130
--tserver_master_addrs=127.15.139.190:40443
--webserver_port=38535
--webserver_interface=127.15.139.130
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.139.148:33791/kms with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 18:02:43 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186563, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:43.790508 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:43.770682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40443 (local address 127.15.139.129:54285)
0923 18:02:43.771277 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:43.771335 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:43.772248 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:43.774259 (+ 2011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:43.774269 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:43.774653 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:43.775404 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:43.775430 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:43.777577 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:43.777581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:43.779046 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:43.779061 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:43.780272 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:43.780276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:43.780494 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:43.781293 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:43.781317 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:43.783138 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:43.786005 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:43.786012 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:43.786031 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:43.786452 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:43.786952 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:43.786956 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:43.786958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:43.787160 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:43.788312 (+ 1152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:43.788320 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:43.788700 (+ 380us) client_negotiation.cc:769] Sending connection context
0923 18:02:43.788925 (+ 225us) client_negotiation.cc:240] Negotiation successful
0923 18:02:43.789317 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":194,"threads_started":1}
I20260923 18:02:43.793215 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:43.771246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:54285 (local address 127.15.139.190:40443)
0923 18:02:43.772784 (+ 1538us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:43.772809 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:43.772863 (+ 54us) server_negotiation.cc:407] Connection header received
0923 18:02:43.773204 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:43.773224 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:43.773712 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:43.774108 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:43.775678 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:43.777374 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:43.779239 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:43.780047 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:43.780917 (+ 870us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:43.783413 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:43.783420 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:43.783441 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:43.783478 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:43.785818 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:43.786628 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:43.786635 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:43.786642 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:43.786802 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:43.787376 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:43.787380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:43.787382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:43.788158 (+ 776us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:43.791552 (+ 3394us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:43.791765 (+ 213us) server_negotiation.cc:299] Negotiation successful
0923 18:02:43.792130 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1325,"thread_start_us":166,"threads_started":1}
I20260923 18:02:43.796430 28903 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40443
I20260923 18:02:43.796720 28903 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:43.797403 28903 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
W20260923 18:02:43.800184 28903 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:40443 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 18:02:43.800347 28903 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:43.800582 28903 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
I20260923 18:02:43.801692 28903 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:43.801887 28903 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
W20260923 18:02:43.802830 28903 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 18:02:43.925143 28907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:43.925498 28907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:43.925570 28907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:43.930444 28907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:43.930541 28907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:43.930648 28907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:02:43.936589 28907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.139.148:33791/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:39811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=38535
--tserver_master_addrs=127.15.139.190:40443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:43.938053 28907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:43.939152 28907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:43.949621 28915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:43.949640 28913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:43.949649 28912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:43.950309 28907 server_base.cc:1034] running on GCE node
I20260923 18:02:43.950775 28907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:43.951629 28907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:43.952806 28907 hybrid_clock.cc:648] HybridClock initialized: now 1790186563952772 us; error 48 us; skew 500 ppm
Sep 23 18:02:43 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186563, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:43.955828 28907 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:02:43.956907 28907 webserver.cc:460] Webserver started at http://127.15.139.130:38535/ using document root <none> and password file <none>
I20260923 18:02:43.957512 28907 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:43.957628 28907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 18:02:43 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186563, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:02:43.976414 28907 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260923 18:02:43.980016 28923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:43.981884 28907 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 18:02:43.982162 28907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "5797e27fed92452db950af253d3a1e05"
format_stamp: "Formatted at 2026-09-23 18:02:42 on dist-test-slave-vr57"
server_key: "fa9c6858af48cd1a2798c03015c5d7ca"
server_key_iv: "c9f72d2672b7af72591f0499d899c8ef"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:43.982990 28907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:44.003348 28907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:44.007649 28907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:44.007880 28907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:44.008606 28907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:44.009743 28907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:02:44.009824 28907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:44.009887 28907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:02:44.009918 28907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:44.021945 28907 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:39811
I20260923 18:02:44.022704 29036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:39811 every 8 connection(s)
I20260923 18:02:44.023319 28907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:02:44.024718 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 28907
I20260923 18:02:44.025199 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.139.148:33791/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:46687
--local_ip_for_outbound_sockets=127.15.139.131
--tserver_master_addrs=127.15.139.190:40443
--webserver_port=46527
--webserver_interface=127.15.139.131
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.139.148:33791/kms with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 18:02:44 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186563, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:44.060297 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:44.027422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:56613 (local address 127.15.139.190:40443)
0923 18:02:44.027598 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:44.027603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:44.036198 (+ 8595us) server_negotiation.cc:407] Connection header received
0923 18:02:44.038064 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:44.038069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:44.038142 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:44.038212 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:44.046807 (+ 8595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:44.047813 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:44.049999 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:44.050396 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:44.050510 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:44.054568 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:44.054574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:44.054577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:44.054613 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:44.056891 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:44.057723 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:44.057727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:44.057730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:44.057843 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:44.058354 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:44.058358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:44.058361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:44.058665 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:44.058776 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:44.059617 (+ 841us) server_negotiation.cc:299] Negotiation successful
0923 18:02:44.059804 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 18:02:44.062541 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:44.026238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40443 (local address 127.15.139.130:56613)
0923 18:02:44.035952 (+ 9714us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:44.036046 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:44.037722 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:44.038725 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:44.038739 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:44.039144 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:44.039885 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:44.039903 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:44.048068 (+ 8165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:44.048073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:44.049792 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:44.049811 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:44.050816 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:44.050821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:44.051072 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:44.052088 (+ 1016us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:44.052117 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:44.054242 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:44.057082 (+ 2840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:44.057090 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:44.057107 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:44.057543 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:44.057988 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:44.057992 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:44.057994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:44.058206 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:44.058863 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:44.058870 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:44.059227 (+ 357us) client_negotiation.cc:769] Sending connection context
0923 18:02:44.060917 (+ 1690us) client_negotiation.cc:240] Negotiation successful
0923 18:02:44.061425 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9383,"thread_start_us":143,"threads_started":1}
I20260923 18:02:44.064404 29037 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40443
I20260923 18:02:44.064656 29037 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:44.065227 29037 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
W20260923 18:02:44.067097 29037 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:40443 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 18:02:44.067217 29037 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:44.067387 29037 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
I20260923 18:02:44.068380 29037 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:44.068547 29037 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
W20260923 18:02:44.069424 29037 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 18:02:44.174103 29040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:44.174405 29040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:44.174449 29040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:44.179001 29040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:44.179069 29040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:44.179167 29040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:02:44.185184 29040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.139.148:33791/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:46687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=46527
--tserver_master_addrs=127.15.139.190:40443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:44.186419 29040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:44.187867 29040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:44.198357 29049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:44.198349 29046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:44.198855 29047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:44.203311 29040 server_base.cc:1034] running on GCE node
I20260923 18:02:44.204031 29040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:44.204965 29040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:44.206243 29040 hybrid_clock.cc:648] HybridClock initialized: now 1790186564206216 us; error 46 us; skew 500 ppm
Sep 23 18:02:44 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186564, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:44.209481 29040 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:02:44.210574 29040 webserver.cc:460] Webserver started at http://127.15.139.131:46527/ using document root <none> and password file <none>
I20260923 18:02:44.211174 29040 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:44.211299 29040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 18:02:44 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186564, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:02:44.230422 29040 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260923 18:02:44.233472 29057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:44.235178 29040 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260923 18:02:44.235348 29040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "b2a64900cc244da8b36d014924965f57"
format_stamp: "Formatted at 2026-09-23 18:02:43 on dist-test-slave-vr57"
server_key: "f0ff9b30db32162d168af58e1070042e"
server_key_iv: "598a99e7cd46e55b8ff2ece3373ca937"
server_key_version: "kuduclusterkey@0"
I20260923 18:02:44.236105 29040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:44.276649 29040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:44.280572 29040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:44.280804 29040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:44.281489 29040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:44.282405 29040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:02:44.282476 29040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:44.282552 29040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:02:44.282601 29040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:44.293397 29040 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:46687
I20260923 18:02:44.294440 29040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:02:44.297725 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 29040
I20260923 18:02:44.308847 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:44.298582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52652 (local address 127.15.139.190:40443)
0923 18:02:44.298730 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:44.298736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:44.299373 (+ 637us) server_negotiation.cc:407] Connection header received
0923 18:02:44.299587 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:44.299591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:44.299647 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:44.299768 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:44.300739 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:44.301546 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:44.302774 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:44.303128 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:44.303259 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:44.305084 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:44.305088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:44.305091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:44.305119 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:44.306821 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:44.307361 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:44.307364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:44.307365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:44.307481 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:44.307894 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:44.307897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:44.307898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:44.308153 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:44.308333 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:44.308573 (+ 240us) server_negotiation.cc:299] Negotiation successful
0923 18:02:44.308677 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260923 18:02:44.310217 29170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:46687 every 8 connection(s)
Sep 23 18:02:44 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186564, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:44.330161 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:44.312645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:50757 (local address 127.15.139.190:40443)
0923 18:02:44.312790 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:44.312795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:44.315270 (+ 2475us) server_negotiation.cc:407] Connection header received
0923 18:02:44.316656 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:44.316662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:44.316718 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:44.316772 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:44.318540 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:44.319352 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:44.321527 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:44.321848 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:44.321927 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:44.325524 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:44.325528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:44.325532 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:44.325560 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:44.327560 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:44.328336 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:44.328339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:44.328342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:44.328436 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:44.328860 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:44.328863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:44.328865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:44.329092 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:44.329176 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:44.329868 (+ 692us) server_negotiation.cc:299] Negotiation successful
0923 18:02:44.329995 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 18:02:44.331840 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:44.313121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40443 (local address 127.15.139.131:50757)
0923 18:02:44.315100 (+ 1979us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:44.315166 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:44.316375 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:44.317205 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:44.317215 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:44.317616 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:44.318344 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:44.318367 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:44.319561 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:44.319564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:44.321363 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:44.321374 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:44.322338 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:44.322344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:44.322572 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:44.323365 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:44.323401 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:44.325294 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:44.327679 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:44.327687 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:44.327707 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:44.328168 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:44.328543 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:44.328546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:44.328548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:44.328744 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:44.329242 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:44.329248 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:44.329571 (+ 323us) client_negotiation.cc:769] Sending connection context
0923 18:02:44.330346 (+ 775us) client_negotiation.cc:240] Negotiation successful
0923 18:02:44.330694 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1739,"thread_start_us":180,"threads_started":1}
I20260923 18:02:44.333145 29171 heartbeater.cc:347] Connected to a master server at 127.15.139.190:40443
I20260923 18:02:44.333427 29171 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:44.333992 29171 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
W20260923 18:02:44.335711 29171 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:40443 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 18:02:44.335882 29171 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:44.336081 29171 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
I20260923 18:02:44.336952 29171 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:44.337150 29171 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
W20260923 18:02:44.338037 29171 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 23 18:02:44 dist-test-slave-vr57 krb5kdc[27709](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790186564, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:44.803099 28903 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:44.803455 28903 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
I20260923 18:02:44.804822 28705 ts_manager.cc:194] Registered new tserver with Master: 79abdac46b3141a9a2a7bb69e166d627 (127.15.139.129:44923)
I20260923 18:02:44.806547 28705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:54285
I20260923 18:02:45.069659 29037 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:45.069996 29037 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
I20260923 18:02:45.070724 28705 ts_manager.cc:194] Registered new tserver with Master: 5797e27fed92452db950af253d3a1e05 (127.15.139.130:39811)
I20260923 18:02:45.071470 28705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:56613
Sep 23 18:02:45 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186564, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:02:45.338271 29171 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:45.338598 29171 heartbeater.cc:511] Master 127.15.139.190:40443 requested a full tablet report, sending...
I20260923 18:02:45.339569 28705 ts_manager.cc:194] Registered new tserver with Master: b2a64900cc244da8b36d014924965f57 (127.15.139.131:46687)
I20260923 18:02:45.340404 28705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:50757
I20260923 18:02:45.342216 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:02:45.357815 29192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.343955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52666 (local address 127.15.139.190:40443)
0923 18:02:45.347719 (+ 3764us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:45.347725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:45.347743 (+ 18us) server_negotiation.cc:407] Connection header received
0923 18:02:45.347791 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:45.347794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:45.347859 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:45.347967 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:45.349000 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.349834 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.351089 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.351441 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.351594 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:45.353371 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:45.353376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:45.353379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:45.353410 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:45.355530 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:45.356085 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:45.356089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:45.356091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:45.356202 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:45.356528 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:45.356531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:45.356533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:45.356804 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:45.356942 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:45.357265 (+ 323us) server_negotiation.cc:299] Negotiation successful
0923 18:02:45.357412 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3674,"thread_start_us":102,"threads_started":1}
I20260923 18:02:45.362389 28705 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:02:45.367595 28705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:02:45.389263 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.381995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56592 (local address 127.15.139.129:44923)
0923 18:02:45.382334 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:45.382342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:45.383112 (+ 770us) server_negotiation.cc:407] Connection header received
0923 18:02:45.383216 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:45.383222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:45.383474 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:45.383694 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:45.384222 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.385411 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.387188 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.388124 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.388231 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:45.388384 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:45.388967 (+ 583us) server_negotiation.cc:299] Negotiation successful
0923 18:02:45.389095 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":101,"threads_started":1}
I20260923 18:02:45.390085 29199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.381822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:44923 (local address 127.0.0.1:56592)
0923 18:02:45.382976 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:45.382993 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:45.383101 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:45.383892 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:45.383899 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:45.383923 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:45.384070 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:45.384077 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.385570 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.385574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:45.387026 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:45.387039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.388560 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.388567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:45.388696 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:45.388744 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 18:02:45.389737 (+ 993us) client_negotiation.cc:240] Negotiation successful
0923 18:02:45.389901 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"thread_start_us":110,"threads_started":1}
I20260923 18:02:45.392386 29201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.382219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:39811 (local address 127.0.0.1:53728)
0923 18:02:45.383284 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:45.383297 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:45.383374 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:45.384139 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:45.384142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:45.384163 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:45.384262 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:45.384269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.387034 (+ 2765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.387038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:45.389424 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:45.389439 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.391954 (+ 2515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.391958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:45.392050 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:45.392078 (+ 28us) client_negotiation.cc:769] Sending connection context
0923 18:02:45.392164 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 18:02:45.392245 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1011,"thread_start_us":101,"threads_started":1}
I20260923 18:02:45.394084 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.382442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.15.139.130:39811)
0923 18:02:45.382748 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:45.382757 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:45.383471 (+ 714us) server_negotiation.cc:407] Connection header received
0923 18:02:45.383605 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:45.383611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:45.383827 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:45.383976 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:45.384416 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.386857 (+ 2441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.389640 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.391594 (+ 1954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.391744 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:45.391882 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:45.393802 (+ 1920us) server_negotiation.cc:299] Negotiation successful
0923 18:02:45.393921 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":142,"threads_started":1}
I20260923 18:02:45.393944 28837 tablet_service.cc:1467] Processing CreateTablet for tablet 0eea49b0c07f4d52b642b40ec0b09be8 (DEFAULT_TABLE table=test-table [id=d01a5152074445e285f97eb52f825137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:02:45.394860 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.380301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:46687 (local address 127.0.0.1:42728)
0923 18:02:45.380780 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:45.380841 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:45.380985 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:45.390570 (+ 9585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:45.390574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:45.390598 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:45.390696 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:45.390702 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.392182 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.392186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:45.393299 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:45.393311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.394463 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.394467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:45.394568 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:45.394594 (+ 26us) client_negotiation.cc:769] Sending connection context
0923 18:02:45.394650 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 18:02:45.394721 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":124,"threads_started":1}
I20260923 18:02:45.395279 28837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eea49b0c07f4d52b642b40ec0b09be8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:45.395591 29198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.381183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42728 (local address 127.15.139.131:46687)
0923 18:02:45.389978 (+ 8795us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:45.389986 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:45.390017 (+ 31us) server_negotiation.cc:407] Connection header received
0923 18:02:45.390108 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:45.390115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:45.390378 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:45.390584 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:45.390867 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.392023 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.393471 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.394308 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.395160 (+ 852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:45.395292 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:45.395342 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 18:02:45.395432 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8651,"thread_start_us":87,"threads_started":1}
I20260923 18:02:45.397215 28971 tablet_service.cc:1467] Processing CreateTablet for tablet 0eea49b0c07f4d52b642b40ec0b09be8 (DEFAULT_TABLE table=test-table [id=d01a5152074445e285f97eb52f825137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:02:45.398175 28971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eea49b0c07f4d52b642b40ec0b09be8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:45.402523 29105 tablet_service.cc:1467] Processing CreateTablet for tablet 0eea49b0c07f4d52b642b40ec0b09be8 (DEFAULT_TABLE table=test-table [id=d01a5152074445e285f97eb52f825137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:02:45.403740 29105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eea49b0c07f4d52b642b40ec0b09be8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:45.417440 29205 tablet_bootstrap.cc:492] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57: Bootstrap starting.
I20260923 18:02:45.418016 29204 tablet_bootstrap.cc:492] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627: Bootstrap starting.
I20260923 18:02:45.420455 29205 tablet_bootstrap.cc:654] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:45.420718 29204 tablet_bootstrap.cc:654] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:45.421473 29205 log.cc:824] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:45.421473 29204 log.cc:824] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:45.422338 29203 tablet_bootstrap.cc:492] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05: Bootstrap starting.
I20260923 18:02:45.424948 29203 tablet_bootstrap.cc:654] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:45.424993 29204 tablet_bootstrap.cc:492] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627: No bootstrap required, opened a new log
I20260923 18:02:45.425364 29204 ts_tablet_manager.cc:1397] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260923 18:02:45.425957 29203 log.cc:824] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:45.428144 29205 tablet_bootstrap.cc:492] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57: No bootstrap required, opened a new log
I20260923 18:02:45.428378 29205 ts_tablet_manager.cc:1397] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260923 18:02:45.431567 29204 raft_consensus.cc:348] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a64900cc244da8b36d014924965f57" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46687 } } peers { permanent_uuid: "79abdac46b3141a9a2a7bb69e166d627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44923 } } peers { permanent_uuid: "5797e27fed92452db950af253d3a1e05" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39811 } }
I20260923 18:02:45.432022 29204 raft_consensus.cc:374] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:45.432127 29204 raft_consensus.cc:729] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79abdac46b3141a9a2a7bb69e166d627, State: Initialized, Role: FOLLOWER
I20260923 18:02:45.432755 29204 consensus_queue.cc:260] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a64900cc244da8b36d014924965f57" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46687 } } peers { permanent_uuid: "79abdac46b3141a9a2a7bb69e166d627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44923 } } peers { permanent_uuid: "5797e27fed92452db950af253d3a1e05" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39811 } }
I20260923 18:02:45.432789 29205 raft_consensus.cc:348] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a64900cc244da8b36d014924965f57" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46687 } } peers { permanent_uuid: "79abdac46b3141a9a2a7bb69e166d627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44923 } } peers { permanent_uuid: "5797e27fed92452db950af253d3a1e05" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39811 } }
I20260923 18:02:45.433094 29205 raft_consensus.cc:374] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:45.433163 29205 raft_consensus.cc:729] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a64900cc244da8b36d014924965f57, State: Initialized, Role: FOLLOWER
I20260923 18:02:45.433660 29205 consensus_queue.cc:260] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a64900cc244da8b36d014924965f57" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46687 } } peers { permanent_uuid: "79abdac46b3141a9a2a7bb69e166d627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44923 } } peers { permanent_uuid: "5797e27fed92452db950af253d3a1e05" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39811 } }
I20260923 18:02:45.434288 29204 ts_tablet_manager.cc:1428] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260923 18:02:45.434724 29205 ts_tablet_manager.cc:1428] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260923 18:02:45.434747 28903 heartbeater.cc:503] Master 127.15.139.190:40443 was elected leader, sending a full tablet report...
I20260923 18:02:45.435093 29171 heartbeater.cc:503] Master 127.15.139.190:40443 was elected leader, sending a full tablet report...
I20260923 18:02:45.436993 29203 tablet_bootstrap.cc:492] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05: No bootstrap required, opened a new log
I20260923 18:02:45.437237 29203 ts_tablet_manager.cc:1397] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260923 18:02:45.441891 29203 raft_consensus.cc:348] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a64900cc244da8b36d014924965f57" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46687 } } peers { permanent_uuid: "79abdac46b3141a9a2a7bb69e166d627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44923 } } peers { permanent_uuid: "5797e27fed92452db950af253d3a1e05" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39811 } }
I20260923 18:02:45.442271 29203 raft_consensus.cc:374] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:45.442342 29203 raft_consensus.cc:729] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5797e27fed92452db950af253d3a1e05, State: Initialized, Role: FOLLOWER
I20260923 18:02:45.443238 29203 consensus_queue.cc:260] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a64900cc244da8b36d014924965f57" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46687 } } peers { permanent_uuid: "79abdac46b3141a9a2a7bb69e166d627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44923 } } peers { permanent_uuid: "5797e27fed92452db950af253d3a1e05" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39811 } }
I20260923 18:02:45.444530 29203 ts_tablet_manager.cc:1428] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260923 18:02:45.444937 29037 heartbeater.cc:503] Master 127.15.139.190:40443 was elected leader, sending a full tablet report...
Sep 23 18:02:45 dist-test-slave-vr57 krb5kdc[27709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186564, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for HTTP/127.15.139.148@KRBTEST.COM
I20260923 18:02:45.502292 29211 raft_consensus.cc:482] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:02:45.502552 29211 raft_consensus.cc:504] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a64900cc244da8b36d014924965f57" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46687 } } peers { permanent_uuid: "79abdac46b3141a9a2a7bb69e166d627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44923 } } peers { permanent_uuid: "5797e27fed92452db950af253d3a1e05" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39811 } }
I20260923 18:02:45.504058 29211 leader_election.cc:288] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79abdac46b3141a9a2a7bb69e166d627 (127.15.139.129:44923), 5797e27fed92452db950af253d3a1e05 (127.15.139.130:39811)
I20260923 18:02:45.516968 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.505466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:48267 (local address 127.15.139.130:39811)
0923 18:02:45.505617 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:45.505623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:45.511908 (+ 6285us) server_negotiation.cc:407] Connection header received
0923 18:02:45.512141 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:45.512144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:45.512196 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:45.512245 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:45.512724 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.513568 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.515629 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.516242 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.516354 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:45.516408 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:45.516741 (+ 333us) server_negotiation.cc:299] Negotiation successful
0923 18:02:45.516822 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 18:02:45.517580 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.505299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:39811 (local address 127.15.139.131:48267)
0923 18:02:45.511809 (+ 6510us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:45.511823 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:45.512005 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:45.512407 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:45.512409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:45.512429 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:45.512575 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:45.512581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.513726 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.513730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:45.515440 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:45.515453 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.516505 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.516508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:45.516606 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:45.516628 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 18:02:45.517209 (+ 581us) client_negotiation.cc:240] Negotiation successful
0923 18:02:45.517291 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6441,"thread_start_us":99,"threads_started":1}
I20260923 18:02:45.518595 28983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eea49b0c07f4d52b642b40ec0b09be8" candidate_uuid: "b2a64900cc244da8b36d014924965f57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5797e27fed92452db950af253d3a1e05" is_pre_election: true
I20260923 18:02:45.518962 28983 raft_consensus.cc:2454] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2a64900cc244da8b36d014924965f57 in term 0.
W20260923 18:02:45.521257 28904 tablet.cc:2369] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:02:45.525478 29038 tablet.cc:2369] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:02:45.526055 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.506216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:59513 (local address 127.15.139.129:44923)
0923 18:02:45.506344 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:45.506349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:45.511637 (+ 5288us) server_negotiation.cc:407] Connection header received
0923 18:02:45.517995 (+ 6358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:45.518000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:45.518067 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:45.518128 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:45.521580 (+ 3452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.522364 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.524602 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.525251 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.525366 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:45.525428 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:45.525815 (+ 387us) server_negotiation.cc:299] Negotiation successful
0923 18:02:45.525906 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 18:02:45.526717 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.504843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:44923 (local address 127.15.139.131:59513)
0923 18:02:45.511437 (+ 6594us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:45.511459 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:45.514761 (+ 3302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:45.521251 (+ 6490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:45.521254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:45.521280 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:45.521412 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:45.521420 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.523099 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.523103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:45.524424 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:45.524438 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.525538 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.525542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:45.525654 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:45.525682 (+ 28us) client_negotiation.cc:769] Sending connection context
0923 18:02:45.526458 (+ 776us) client_negotiation.cc:240] Negotiation successful
0923 18:02:45.526546 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6493,"thread_start_us":133,"threads_started":1}
I20260923 18:02:45.527608 28857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eea49b0c07f4d52b642b40ec0b09be8" candidate_uuid: "b2a64900cc244da8b36d014924965f57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79abdac46b3141a9a2a7bb69e166d627" is_pre_election: true
I20260923 18:02:45.527973 28857 raft_consensus.cc:2454] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2a64900cc244da8b36d014924965f57 in term 0.
I20260923 18:02:45.528122 29059 leader_election.cc:302] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5797e27fed92452db950af253d3a1e05, b2a64900cc244da8b36d014924965f57; no voters:
I20260923 18:02:45.528468 29211 raft_consensus.cc:2781] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:02:45.528586 29211 raft_consensus.cc:482] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:02:45.528643 29211 raft_consensus.cc:3037] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:45.530136 29211 raft_consensus.cc:504] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a64900cc244da8b36d014924965f57" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46687 } } peers { permanent_uuid: "79abdac46b3141a9a2a7bb69e166d627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44923 } } peers { permanent_uuid: "5797e27fed92452db950af253d3a1e05" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39811 } }
I20260923 18:02:45.530591 29211 leader_election.cc:288] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [CANDIDATE]: Term 1 election: Requested vote from peers 79abdac46b3141a9a2a7bb69e166d627 (127.15.139.129:44923), 5797e27fed92452db950af253d3a1e05 (127.15.139.130:39811)
I20260923 18:02:45.531852 28983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eea49b0c07f4d52b642b40ec0b09be8" candidate_uuid: "b2a64900cc244da8b36d014924965f57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5797e27fed92452db950af253d3a1e05"
I20260923 18:02:45.531864 28857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eea49b0c07f4d52b642b40ec0b09be8" candidate_uuid: "b2a64900cc244da8b36d014924965f57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79abdac46b3141a9a2a7bb69e166d627"
I20260923 18:02:45.532012 28857 raft_consensus.cc:3037] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:45.532014 28983 raft_consensus.cc:3037] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:45.533407 28857 raft_consensus.cc:2454] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2a64900cc244da8b36d014924965f57 in term 1.
I20260923 18:02:45.533416 28983 raft_consensus.cc:2454] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2a64900cc244da8b36d014924965f57 in term 1.
I20260923 18:02:45.533906 29060 leader_election.cc:302] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79abdac46b3141a9a2a7bb69e166d627, b2a64900cc244da8b36d014924965f57; no voters:
I20260923 18:02:45.534449 29211 raft_consensus.cc:2781] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:02:45.534719 29211 raft_consensus.cc:686] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 1 LEADER]: Becoming Leader. State: Replica: b2a64900cc244da8b36d014924965f57, State: Running, Role: LEADER
I20260923 18:02:45.535104 29211 consensus_queue.cc:237] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a64900cc244da8b36d014924965f57" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46687 } } peers { permanent_uuid: "79abdac46b3141a9a2a7bb69e166d627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44923 } } peers { permanent_uuid: "5797e27fed92452db950af253d3a1e05" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39811 } }
I20260923 18:02:45.540107 28704 catalog_manager.cc:5495] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2a64900cc244da8b36d014924965f57 (127.15.139.131). New cstate: current_term: 1 leader_uuid: "b2a64900cc244da8b36d014924965f57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a64900cc244da8b36d014924965f57" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 46687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79abdac46b3141a9a2a7bb69e166d627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5797e27fed92452db950af253d3a1e05" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39811 } health_report { overall_health: UNKNOWN } } }
W20260923 18:02:45.562314 29172 tablet.cc:2369] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:02:45.587636 29198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.582052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42734 (local address 127.15.139.131:46687)
0923 18:02:45.582224 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:45.582230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:45.582422 (+ 192us) server_negotiation.cc:407] Connection header received
0923 18:02:45.582526 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:45.582530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:45.582606 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:45.582716 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:02:45.583137 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.583993 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.585271 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:45.585622 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:45.585726 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:45.586205 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:02:45.586789 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:02:45.586945 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:45.587385 (+ 440us) server_negotiation.cc:299] Negotiation successful
0923 18:02:45.587450 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260923 18:02:45.603609 28983 raft_consensus.cc:1261] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05 [term 1 FOLLOWER]: Refusing update from remote peer b2a64900cc244da8b36d014924965f57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:02:45.603709 28857 raft_consensus.cc:1261] T 0eea49b0c07f4d52b642b40ec0b09be8 P 79abdac46b3141a9a2a7bb69e166d627 [term 1 FOLLOWER]: Refusing update from remote peer b2a64900cc244da8b36d014924965f57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:02:45.604449 29216 consensus_queue.cc:1035] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5797e27fed92452db950af253d3a1e05" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 39811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:02:45.605211 29216 consensus_queue.cc:1035] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79abdac46b3141a9a2a7bb69e166d627" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:02:45.634671 29222 mvcc.cc:205] Tried to move back new op lower bound from 7332604172701065216 to 7332604172440367104. Current Snapshot: MvccSnapshot[applied={T|T < 7332604172701065216}]
I20260923 18:02:45.646514 28704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:02:45.646726 28704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:02:45.649646 28704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9174b54cc00a4ee68764cd38bbe2a799: Sending DeleteTablet for 3 replicas of tablet 0eea49b0c07f4d52b642b40ec0b09be8
I20260923 18:02:45.651275 29105 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eea49b0c07f4d52b642b40ec0b09be8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:02:45 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:42728
I20260923 18:02:45.651875 28837 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eea49b0c07f4d52b642b40ec0b09be8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:02:45 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:56592
I20260923 18:02:45.652424 28971 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eea49b0c07f4d52b642b40ec0b09be8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:02:45 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:53728
I20260923 18:02:45.653942 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 28743
I20260923 18:02:45.658018 29228 tablet_replica.cc:330] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57: stopping tablet replica
I20260923 18:02:45.658370 29228 raft_consensus.cc:2229] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:02:45.658793 29228 raft_consensus.cc:2258] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:02:45.659226 29230 tablet_replica.cc:330] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05: stopping tablet replica
I20260923 18:02:45.659580 29230 raft_consensus.cc:2229] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:02:45.659860 29230 raft_consensus.cc:2258] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:02:45.671142 29228 ts_tablet_manager.cc:1905] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:02:45.675163 29228 ts_tablet_manager.cc:1918] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:02:45.675294 29228 log.cc:1196] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/0eea49b0c07f4d52b642b40ec0b09be8
I20260923 18:02:45.680708 29230 ts_tablet_manager.cc:1905] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:02:45.683626 29230 ts_tablet_manager.cc:1918] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:02:45.683740 29230 log.cc:1196] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/0eea49b0c07f4d52b642b40ec0b09be8
I20260923 18:02:45.684118 29230 ts_tablet_manager.cc:1939] T 0eea49b0c07f4d52b642b40ec0b09be8 P 5797e27fed92452db950af253d3a1e05: Deleting consensus metadata
I20260923 18:02:45.685912 28690 catalog_manager.cc:4841] TS 5797e27fed92452db950af253d3a1e05 (127.15.139.130:39811): tablet 0eea49b0c07f4d52b642b40ec0b09be8 (table test-table [id=d01a5152074445e285f97eb52f825137]) successfully deleted
I20260923 18:02:45.686108 29228 ts_tablet_manager.cc:1939] T 0eea49b0c07f4d52b642b40ec0b09be8 P b2a64900cc244da8b36d014924965f57: Deleting consensus metadata
W20260923 18:02:45.687211 28691 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:44923 (error 108)
I20260923 18:02:45.688721 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:45.688391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:44923 (local address 127.0.0.1:56606)
0923 18:02:45.688540 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:45.688617 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:44923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260923 18:02:45.688854 28691 catalog_manager.cc:4584] TS 79abdac46b3141a9a2a7bb69e166d627 (127.15.139.129:44923): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0eea49b0c07f4d52b642b40ec0b09be8: Network error: Client connection negotiation failed: client connection to 127.15.139.129:44923: connect: Connection refused (error 111)
I20260923 18:02:45.689208 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 28907
I20260923 18:02:45.689517 28689 catalog_manager.cc:4841] TS b2a64900cc244da8b36d014924965f57 (127.15.139.131:46687): tablet 0eea49b0c07f4d52b642b40ec0b09be8 (table test-table [id=d01a5152074445e285f97eb52f825137]) successfully deleted
I20260923 18:02:45.698580 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 29040
I20260923 18:02:45.708036 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 28672
I20260923 18:02:45.718865 15918 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 18:02:46.064481 15918 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 18:02:46.064639 15918 mini_ranger.cc:66] Stopping Ranger...
I20260923 18:02:46.410712 15918 mini_ranger.cc:68] Stopped Ranger
2026-09-23 18:02:46.410 UTC [27739] LOG: received smart shutdown request
2026-09-23 18:02:46.414 UTC [27739] LOG: background worker "logical replication launcher" (PID 27748) exited with exit code 1
2026-09-23 18:02:46.415 UTC [27743] LOG: shutting down
2026-09-23 18:02:46.605 UTC [27739] LOG: database system is shut down
2026-09-23T18:02:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50754 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:02:46 dist-test-slave-vr57 krb5kdc[29240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:02:46 dist-test-slave-vr57 krb5kdc[29240](info): set up 2 sockets
Sep 23 18:02:46 dist-test-slave-vr57 krb5kdc[29240](info): commencing operation
krb5kdc: starting...
W20260923 18:02:47.804744 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:02:47 dist-test-slave-vr57 krb5kdc[29240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:02:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:02:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:02:47.932961 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:33899
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:33899,127.15.139.189:40103,127.15.139.188:40847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:48.047562 29253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:48.047889 29253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:48.047960 29253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:48.052026 29253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:02:48.052115 29253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:48.052153 29253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:48.052248 29253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:02:48.052304 29253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:02:48.057688 29253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:33899,127.15.139.189:40103,127.15.139.188:40847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:33899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:48.058846 29253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:48.060091 29253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:48.069895 29259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:48.070031 29253 server_base.cc:1034] running on GCE node
W20260923 18:02:48.069876 29258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:48.069927 29261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:48.070757 29253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:48.071923 29253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:48.073127 29253 hybrid_clock.cc:648] HybridClock initialized: now 1790186568073091 us; error 49 us; skew 500 ppm
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186568, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:48.079746 29253 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:02:48.080786 29253 webserver.cc:460] Webserver started at http://127.15.139.190:33655/ using document root <none> and password file <none>
I20260923 18:02:48.081382 29253 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:48.081475 29253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:48.081725 29253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:02:48.083938 29253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "2b8271528e3646168499979c50716dea"
format_stamp: "Formatted at 2026-09-23 18:02:48 on dist-test-slave-vr57"
server_key: "f84a3a1de5f36ff071bf576dc0713281"
server_key_iv: "6f66a7e196d7f6ec1a54a6404c562bdb"
server_key_version: "encryptionkey@0"
I20260923 18:02:48.084528 29253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "2b8271528e3646168499979c50716dea"
format_stamp: "Formatted at 2026-09-23 18:02:48 on dist-test-slave-vr57"
server_key: "f84a3a1de5f36ff071bf576dc0713281"
server_key_iv: "6f66a7e196d7f6ec1a54a6404c562bdb"
server_key_version: "encryptionkey@0"
I20260923 18:02:48.088810 29253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260923 18:02:48.091384 29268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:48.092707 29253 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260923 18:02:48.092872 29253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "2b8271528e3646168499979c50716dea"
format_stamp: "Formatted at 2026-09-23 18:02:48 on dist-test-slave-vr57"
server_key: "f84a3a1de5f36ff071bf576dc0713281"
server_key_iv: "6f66a7e196d7f6ec1a54a6404c562bdb"
server_key_version: "encryptionkey@0"
I20260923 18:02:48.092988 29253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:48.132088 29253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:48.135600 29253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:48.135883 29253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:48.143610 29253 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:33899
I20260923 18:02:48.143940 29320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:33899 every 8 connection(s)
I20260923 18:02:48.144693 29253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:02:48.147807 29321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:48.150652 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 29253
I20260923 18:02:48.150820 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:02:48.151124 15918 external_mini_cluster.cc:1428] Setting key d2601037cfd945da5b957d47ea5b18ab
I20260923 18:02:48.151965 29321 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.139.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.189@KRBTEST.COM" created.
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186568, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:48.168576 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.154571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:33899 (local address 127.0.0.1:49576)
0923 18:02:48.155218 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.155266 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:48.156394 (+ 1128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:48.157115 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:48.157120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:48.157594 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:48.157849 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.157863 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.159029 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.159032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.159899 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.159909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.160582 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.160586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.160685 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.161338 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:48.161356 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:48.163122 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:48.165309 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.165320 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.165331 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.165569 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.165910 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.165913 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.165915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.166029 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.166686 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:48.166693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:48.166845 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 18:02:48.167042 (+ 197us) client_negotiation.cc:240] Negotiation successful
0923 18:02:48.167466 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":121,"spinlock_wait_cycles":11648,"thread_start_us":167,"threads_started":1}
I20260923 18:02:48.168576 29325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.155721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.15.139.190:33899)
0923 18:02:48.156065 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.156076 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.156123 (+ 47us) server_negotiation.cc:407] Connection header received
0923 18:02:48.156723 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.156728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.156924 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.157117 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.158000 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.158884 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.160018 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.160419 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.160598 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.163324 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.163337 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.163350 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.163373 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.165170 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.165688 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.165694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.165696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.165806 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.166131 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.166134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.166136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.166563 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.166709 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.167106 (+ 397us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.167410 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":232,"thread_start_us":135,"threads_started":1}
I20260923 18:02:48.171475 29321 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:02:48.172348 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.172074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:40103 (local address 127.0.0.1:48822)
0923 18:02:48.172195 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.172279 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:40103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 18:02:48.172626 29269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.139.189:40103: connect: Connection refused (error 111)
I20260923 18:02:48.173588 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.173369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:40103 (local address 127.0.0.1:48836)
0923 18:02:48.173479 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.173525 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:40103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 18:02:48.173779 29321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.189:40103: Network error: Client connection negotiation failed: client connection to 127.15.139.189:40103: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.189.keytab.
Entry for principal kudu/127.15.139.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.189.keytab.
I20260923 18:02:48.178835 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.189:40103
--webserver_interface=127.15.139.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.189.keytab
--principal=kudu/127.15.139.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:33899,127.15.139.189:40103,127.15.139.188:40847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:02:48.223064 29321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } attempt: 1
I20260923 18:02:48.224092 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.223740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:40103 (local address 127.0.0.1:48850)
0923 18:02:48.223918 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.223995 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:40103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260923 18:02:48.224922 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.224694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:40103 (local address 127.0.0.1:48858)
0923 18:02:48.224809 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.224861 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:40103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260923 18:02:48.225066 29321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.189:40103: Network error: Client connection negotiation failed: client connection to 127.15.139.189:40103: connect: Connection refused (error 111)
I20260923 18:02:48.293260 29321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } attempt: 2
I20260923 18:02:48.294135 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.293795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:40103 (local address 127.0.0.1:48866)
0923 18:02:48.293966 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.294041 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:40103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260923 18:02:48.294983 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.294755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:40103 (local address 127.0.0.1:48872)
0923 18:02:48.294863 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.294916 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:40103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260923 18:02:48.295145 29321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.189:40103: Network error: Client connection negotiation failed: client connection to 127.15.139.189:40103: connect: Connection refused (error 111)
W20260923 18:02:48.295701 29328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:48.295975 29328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:48.296067 29328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:48.300325 29328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:02:48.300383 29328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:48.300403 29328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:48.300420 29328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:02:48.300469 29328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:02:48.306141 29328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:33899,127.15.139.189:40103,127.15.139.188:40847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.189.keytab
--principal=kudu/127.15.139.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.189:40103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.139.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:48.307322 29328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:48.308432 29328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:48.317498 29334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:48.317404 29333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:48.317374 29336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:48.317898 29328 server_base.cc:1034] running on GCE node
I20260923 18:02:48.318447 29328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:48.319832 29328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:48.321051 29328 hybrid_clock.cc:648] HybridClock initialized: now 1790186568321006 us; error 54 us; skew 500 ppm
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186568, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:48.324046 29328 init.cc:377] Logged in from keytab as kudu/127.15.139.189@KRBTEST.COM (short username kudu)
I20260923 18:02:48.325052 29328 webserver.cc:460] Webserver started at http://127.15.139.189:45155/ using document root <none> and password file <none>
I20260923 18:02:48.325623 29328 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:48.325714 29328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:48.325919 29328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:02:48.328087 29328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/data/instance:
uuid: "26a840a6a8c24346ad14960fb759b954"
format_stamp: "Formatted at 2026-09-23 18:02:48 on dist-test-slave-vr57"
server_key: "94cb0e13847e7de89edfc40900388584"
server_key_iv: "9773aa1d045f6ab3ead96f984cbf2a60"
server_key_version: "encryptionkey@0"
I20260923 18:02:48.328679 29328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/wal/instance:
uuid: "26a840a6a8c24346ad14960fb759b954"
format_stamp: "Formatted at 2026-09-23 18:02:48 on dist-test-slave-vr57"
server_key: "94cb0e13847e7de89edfc40900388584"
server_key_iv: "9773aa1d045f6ab3ead96f984cbf2a60"
server_key_version: "encryptionkey@0"
I20260923 18:02:48.333077 29328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:02:48.335831 29343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:48.337070 29328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:02:48.337268 29328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/wal
uuid: "26a840a6a8c24346ad14960fb759b954"
format_stamp: "Formatted at 2026-09-23 18:02:48 on dist-test-slave-vr57"
server_key: "94cb0e13847e7de89edfc40900388584"
server_key_iv: "9773aa1d045f6ab3ead96f984cbf2a60"
server_key_version: "encryptionkey@0"
I20260923 18:02:48.337389 29328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:48.355235 29328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:48.358704 29328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:48.358865 29328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:48.365974 29328 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.189:40103
I20260923 18:02:48.366323 29395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.189:40103 every 8 connection(s)
I20260923 18:02:48.367017 29328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
I20260923 18:02:48.369962 29396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:48.373785 29396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:02:48.375896 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 29328
I20260923 18:02:48.376029 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-1/wal/instance
I20260923 18:02:48.376354 15918 external_mini_cluster.cc:1428] Setting key bee12439ae5457c2b4f5ee232a12afae
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186568, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.189@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.188@KRBTEST.COM; defaulting to no policy
I20260923 18:02:48.386348 29321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.188@KRBTEST.COM" created.
I20260923 18:02:48.389554 29325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.375851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49592 (local address 127.15.139.190:33899)
0923 18:02:48.375998 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.376004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.377077 (+ 1073us) server_negotiation.cc:407] Connection header received
0923 18:02:48.378004 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.378008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.378078 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.378163 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.379570 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.380131 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.381998 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.382257 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.382346 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.385379 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.385382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.385384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.385419 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.387266 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.387904 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.387911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.387914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.388007 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.388429 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.388432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.388434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.388616 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.388728 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.389255 (+ 527us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.389375 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 18:02:48.390971 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.376385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:33899 (local address 127.0.0.1:49592)
0923 18:02:48.376922 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.376966 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:48.377783 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:48.378351 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:48.378359 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:48.378697 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:48.379350 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.379372 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.380273 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.380280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.381851 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.381865 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.382428 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.382434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.382664 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.383358 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:48.383386 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:48.385118 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:48.387381 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.387386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.387402 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.387790 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.388123 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.388127 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.388128 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.388312 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.388732 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:48.388741 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:48.388998 (+ 257us) client_negotiation.cc:769] Sending connection context
0923 18:02:48.389193 (+ 195us) client_negotiation.cc:240] Negotiation successful
0923 18:02:48.389969 (+ 776us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":33,"thread_start_us":155,"threads_started":1}
I20260923 18:02:48.392802 29396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186568, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.189@KRBTEST.COM
I20260923 18:02:48.398200 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.386788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:40103 (local address 127.0.0.1:48886)
0923 18:02:48.386930 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.386947 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:48.387033 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:48.387955 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:48.387961 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:48.388135 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:48.388276 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.388283 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.389805 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.389810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.390599 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.390609 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.391663 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.391670 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.391812 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.392471 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:48.392491 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:48.394125 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:48.396419 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.396424 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.396428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.396676 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.397026 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.397033 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.397038 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.397140 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.397668 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:48.397671 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:48.397776 (+ 105us) client_negotiation.cc:769] Sending connection context
0923 18:02:48.397930 (+ 154us) client_negotiation.cc:240] Negotiation successful
0923 18:02:48.398071 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260923 18:02:48.398936 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.386982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48886 (local address 127.15.139.189:40103)
0923 18:02:48.387431 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.387441 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.387462 (+ 21us) server_negotiation.cc:407] Connection header received
0923 18:02:48.387549 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.387555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.387785 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.387980 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.388405 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.389674 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.390729 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.391357 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.391480 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.394272 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.394282 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.394295 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.394315 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.396273 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.396795 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.396804 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.396808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.396923 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.397285 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.397290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.397292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.397556 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.397703 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.398473 (+ 770us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.398595 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":338,"spinlock_wait_cycles":18048,"thread_start_us":187,"threads_started":1}
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186568, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.189@KRBTEST.COM for kudu/127.15.139.189@KRBTEST.COM
I20260923 18:02:48.401086 29321 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:02:48.401906 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.401626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:41898)
0923 18:02:48.401763 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.401828 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 18:02:48.402751 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.402486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:41910)
0923 18:02:48.402637 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.402696 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260923 18:02:48.402952 29321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:40847: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
I20260923 18:02:48.403960 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.393395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:40103 (local address 127.0.0.1:48898)
0923 18:02:48.393538 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.393552 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:48.393632 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:48.394097 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:48.394100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:48.394282 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:48.394393 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.394401 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.395562 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.395566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.396420 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.396430 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.396893 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.396896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.397010 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.397417 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:48.397429 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:48.400026 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:48.402417 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.402422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.402424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.402614 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.402934 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.402937 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.402941 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.403014 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.403394 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:48.403397 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:48.403481 (+ 84us) client_negotiation.cc:769] Sending connection context
0923 18:02:48.403630 (+ 149us) client_negotiation.cc:240] Negotiation successful
0923 18:02:48.403774 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260923 18:02:48.403983 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.393599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.15.139.189:40103)
0923 18:02:48.393901 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.393917 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.393928 (+ 11us) server_negotiation.cc:407] Connection header received
0923 18:02:48.393954 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.393956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.393992 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.394088 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.394527 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.395408 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.396505 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.396780 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.397229 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.400180 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.400184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.400191 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.400217 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.402320 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.402712 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.402718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.402720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.402827 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.403112 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.403115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.403116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.403318 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.403406 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.403687 (+ 281us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.403831 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":242,"thread_start_us":106,"threads_started":1}
I20260923 18:02:48.404844 29396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:02:48.405611 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.405310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:41922)
0923 18:02:48.405477 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.405555 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260923 18:02:48.405833 29347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.188.keytab.
Entry for principal kudu/127.15.139.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.188.keytab.
I20260923 18:02:48.407009 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.406710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:41938)
0923 18:02:48.406839 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.406893 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260923 18:02:48.407177 29396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:40847: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
I20260923 18:02:48.407945 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.188:40847
--webserver_interface=127.15.139.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.188.keytab
--principal=kudu/127.15.139.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:33899,127.15.139.189:40103,127.15.139.188:40847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:02:48.434137 29321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } attempt: 1
I20260923 18:02:48.435178 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.434772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:41952)
0923 18:02:48.434999 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.435076 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260923 18:02:48.436076 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.435822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:41968)
0923 18:02:48.435948 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.436005 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260923 18:02:48.436267 29321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:40847: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
I20260923 18:02:48.456493 29396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } attempt: 1
I20260923 18:02:48.457444 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.457136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:41974)
0923 18:02:48.457298 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.457362 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260923 18:02:48.458283 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.458081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:41978)
0923 18:02:48.458179 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.458225 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260923 18:02:48.458544 29396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:40847: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
I20260923 18:02:48.511456 29321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } attempt: 2
I20260923 18:02:48.512557 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.512078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:41980)
0923 18:02:48.512295 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.512450 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260923 18:02:48.513778 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.513551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:41986)
0923 18:02:48.513671 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.513725 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260923 18:02:48.513986 29321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:40847: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
W20260923 18:02:48.524529 29404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:48.524829 29404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:48.524928 29404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260923 18:02:48.526705 29396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } attempt: 2
I20260923 18:02:48.527609 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.527235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:42002)
0923 18:02:48.527411 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.527472 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260923 18:02:48.528327 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.528110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:42004)
0923 18:02:48.528219 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.528273 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260923 18:02:48.528467 29396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:40847: Network error: Client connection negotiation failed: client connection to 127.15.139.188:40847: connect: Connection refused (error 111)
W20260923 18:02:48.529505 29404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:02:48.529557 29404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:48.529599 29404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:48.529635 29404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:02:48.529680 29404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:02:48.535163 29404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:33899,127.15.139.189:40103,127.15.139.188:40847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.188.keytab
--principal=kudu/127.15.139.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.188:40847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.139.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:48.536276 29404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:48.537302 29404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:48.545874 29412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:48.545859 29410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:48.545930 29404 server_base.cc:1034] running on GCE node
W20260923 18:02:48.545854 29409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:48.546598 29404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:48.547679 29404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:48.548839 29404 hybrid_clock.cc:648] HybridClock initialized: now 1790186568548821 us; error 28 us; skew 500 ppm
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186568, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:48.556391 29404 init.cc:377] Logged in from keytab as kudu/127.15.139.188@KRBTEST.COM (short username kudu)
I20260923 18:02:48.557374 29404 webserver.cc:460] Webserver started at http://127.15.139.188:38181/ using document root <none> and password file <none>
I20260923 18:02:48.557998 29404 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:48.558110 29404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:48.558351 29404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:02:48.560524 29404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/data/instance:
uuid: "d1dc51bbfd3141b69d49e432ea7da9b8"
format_stamp: "Formatted at 2026-09-23 18:02:48 on dist-test-slave-vr57"
server_key: "ac8ca8eec8c3bea11cc2f1894d87ab2d"
server_key_iv: "80f216cf502a3d5d4006c11b6af38826"
server_key_version: "encryptionkey@0"
I20260923 18:02:48.561096 29404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/wal/instance:
uuid: "d1dc51bbfd3141b69d49e432ea7da9b8"
format_stamp: "Formatted at 2026-09-23 18:02:48 on dist-test-slave-vr57"
server_key: "ac8ca8eec8c3bea11cc2f1894d87ab2d"
server_key_iv: "80f216cf502a3d5d4006c11b6af38826"
server_key_version: "encryptionkey@0"
I20260923 18:02:48.564836 29404 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:02:48.567212 29419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:48.568437 29404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:02:48.568578 29404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/wal
uuid: "d1dc51bbfd3141b69d49e432ea7da9b8"
format_stamp: "Formatted at 2026-09-23 18:02:48 on dist-test-slave-vr57"
server_key: "ac8ca8eec8c3bea11cc2f1894d87ab2d"
server_key_iv: "80f216cf502a3d5d4006c11b6af38826"
server_key_version: "encryptionkey@0"
I20260923 18:02:48.568689 29404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:48.584765 29404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:48.588486 29404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:48.588688 29404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:48.595884 29404 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.188:40847
I20260923 18:02:48.596222 29471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.188:40847 every 8 connection(s)
I20260923 18:02:48.596980 29404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
I20260923 18:02:48.599869 29472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:48.603590 29472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:02:48.605459 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 29404
I20260923 18:02:48.605577 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790186398863621-15918-0/minicluster-data/master-2/wal/instance
I20260923 18:02:48.605862 15918 external_mini_cluster.cc:1428] Setting key 86a682c4e2e9948b36e8dba367ad8107
I20260923 18:02:48.613168 29321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } attempt: 3
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186568, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.188@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.188@KRBTEST.COM
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186568, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.188@KRBTEST.COM
I20260923 18:02:48.619575 29325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.605452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.15.139.190:33899)
0923 18:02:48.605586 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.605589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.607219 (+ 1630us) server_negotiation.cc:407] Connection header received
0923 18:02:48.608049 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.608052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.608102 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.608193 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.609296 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.610082 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.611537 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.611878 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.612000 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.615025 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.615029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.615031 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.615061 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.616754 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.617445 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.617448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.617450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.617529 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.618389 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.618393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.618395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.618584 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.618694 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.619228 (+ 534us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.619364 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 18:02:48.619630 29396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } attempt: 3
I20260923 18:02:48.621186 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.606577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42010 (local address 127.15.139.188:40847)
0923 18:02:48.607006 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.607020 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.607073 (+ 53us) server_negotiation.cc:407] Connection header received
0923 18:02:48.607814 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.607831 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.608063 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.608241 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.609155 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.610370 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.611539 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.612099 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.612196 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.615371 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.615378 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.615393 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.615434 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.617510 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.617958 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.617964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.617966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.618108 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.618454 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.618459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.618464 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.618988 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.619190 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.619452 (+ 262us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.620052 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":323,"spinlock_wait_cycles":3456,"thread_start_us":133,"threads_started":1}
I20260923 18:02:48.621189 29474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.605922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:33899 (local address 127.0.0.1:49600)
0923 18:02:48.607073 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.607123 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:48.607823 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:48.608259 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:48.608265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:48.608757 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:48.609163 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.609176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.610212 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.610215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.611399 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.611408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.612033 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.612036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.612193 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.612866 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:48.612904 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:48.614872 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:48.616888 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.616897 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.616907 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.617312 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.618139 (+ 827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.618145 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.618150 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.618282 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.618703 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:48.618713 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:48.618968 (+ 255us) client_negotiation.cc:769] Sending connection context
0923 18:02:48.620024 (+ 1056us) client_negotiation.cc:240] Negotiation successful
0923 18:02:48.620527 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"mutex_wait_us":1,"spinlock_wait_cycles":3840,"thread_start_us":156,"threads_started":1}
I20260923 18:02:48.622998 29472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:02:48.623569 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.613628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:42018)
0923 18:02:48.613822 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.613838 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:48.613918 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:48.614226 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:48.614229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:48.614469 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:48.614565 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.614572 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.615629 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.615634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.616481 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.616495 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.617100 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.617117 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.617213 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.617673 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:48.617685 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:48.619112 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:48.621968 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.621972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.621975 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.622232 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.622568 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.622571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.622573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.622657 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.623088 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:48.623091 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:48.623184 (+ 93us) client_negotiation.cc:769] Sending connection context
0923 18:02:48.623263 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 18:02:48.623389 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260923 18:02:48.624210 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.613662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42018 (local address 127.15.139.188:40847)
0923 18:02:48.614031 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.614039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.614049 (+ 10us) server_negotiation.cc:407] Connection header received
0923 18:02:48.614075 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.614076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.614109 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.614206 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.614696 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.615491 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.616625 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.616974 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.617087 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.619256 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.619259 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.619261 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.619279 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.621789 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.622347 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.622350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.622351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.622458 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.622763 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.622766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.622767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.622969 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.623839 (+ 870us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.623935 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.624047 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":107,"threads_started":1}
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186568, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.189@KRBTEST.COM for kudu/127.15.139.188@KRBTEST.COM
I20260923 18:02:48.628985 29321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea: Bootstrap starting.
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186568, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.188@KRBTEST.COM for kudu/127.15.139.189@KRBTEST.COM
I20260923 18:02:48.630589 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.620079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:42028)
0923 18:02:48.620185 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.620200 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:48.620291 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:48.621795 (+ 1504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:48.621798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:48.621993 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:48.622092 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.622099 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.623056 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.623060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.623844 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.623853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.624503 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.624509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.624598 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.625151 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:48.625165 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:48.626671 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:48.628806 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.628813 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.628815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.629076 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.629428 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.629432 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.629434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.629534 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.630008 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:48.630011 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:48.630112 (+ 101us) client_negotiation.cc:769] Sending connection context
0923 18:02:48.630212 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 18:02:48.630377 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:48.631124 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.620323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42028 (local address 127.15.139.188:40847)
0923 18:02:48.621567 (+ 1244us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.621571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.621584 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:02:48.621616 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.621619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.621655 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.621742 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.622241 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.622928 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.623973 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.624304 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.624392 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.626804 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.626807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.626809 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.626833 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.628671 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.629186 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.629192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.629197 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.629294 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.629643 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.629646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.629648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.629862 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.629956 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.630818 (+ 862us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.630953 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1187,"spinlock_wait_cycles":1920,"thread_start_us":129,"threads_started":1}
I20260923 18:02:48.633634 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.623704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48904 (local address 127.15.139.189:40103)
0923 18:02:48.623802 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.623807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.624098 (+ 291us) server_negotiation.cc:407] Connection header received
0923 18:02:48.624236 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.624238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.624282 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.624345 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.624867 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.625634 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.626787 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.627209 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.627309 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.629965 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.629969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.629974 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.630003 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.631703 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.632212 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.632215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.632217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.632324 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.632638 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.632640 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.632642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.632896 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.632972 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.633365 (+ 393us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.633481 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 18:02:48.634109 29474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.623468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:40103 (local address 127.0.0.1:48904)
0923 18:02:48.624041 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.624054 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:48.624146 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:48.624477 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:48.624480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:48.624643 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:48.624762 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.624773 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.625767 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.625773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.626655 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.626665 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.627717 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.627721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.627822 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.628252 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:48.628267 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:48.629772 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:48.631856 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.631859 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.631861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.632093 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.632447 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.632450 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.632452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.632536 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.633050 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:48.633056 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:48.633161 (+ 105us) client_negotiation.cc:769] Sending connection context
0923 18:02:48.633807 (+ 646us) client_negotiation.cc:240] Negotiation successful
0923 18:02:48.633969 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"spinlock_wait_cycles":1920}
I20260923 18:02:48.635049 29472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:02:48.635408 29396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954: Bootstrap starting.
I20260923 18:02:48.637342 29321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:48.638373 29325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.624976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49612 (local address 127.15.139.190:33899)
0923 18:02:48.625084 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.625088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.625306 (+ 218us) server_negotiation.cc:407] Connection header received
0923 18:02:48.625491 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.625494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.625534 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.625581 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.626400 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.627105 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.628302 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.628611 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.628687 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.631720 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.631724 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.631727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.631753 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.635908 (+ 4155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.636442 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.636446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.636448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.636533 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.637005 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.637008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.637010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.637250 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.637433 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.637945 (+ 512us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.638083 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 18:02:48.638816 29396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:48.638830 29321 log.cc:824] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:48.639720 29396 log.cc:824] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:48.641856 29321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea: No bootstrap required, opened a new log
I20260923 18:02:48.642268 29396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954: No bootstrap required, opened a new log
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186568, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.188@KRBTEST.COM for kudu/127.15.139.188@KRBTEST.COM
I20260923 18:02:48.647357 29396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } }
I20260923 18:02:48.647660 29396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:48.647735 29396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26a840a6a8c24346ad14960fb759b954, State: Initialized, Role: FOLLOWER
I20260923 18:02:48.648363 29396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } }
I20260923 18:02:48.649570 29321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } }
I20260923 18:02:48.649878 29321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:48.649930 29396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:02:48.650002 29321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b8271528e3646168499979c50716dea, State: Initialized, Role: FOLLOWER
I20260923 18:02:48.650576 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.638111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42030 (local address 127.15.139.188:40847)
0923 18:02:48.638213 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.638217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.640272 (+ 2055us) server_negotiation.cc:407] Connection header received
0923 18:02:48.640364 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.640367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.640407 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.640508 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.640935 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.641695 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.642751 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.643038 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.643111 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.645914 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.645918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.645921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.645943 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.648426 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.649309 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.649312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.649314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.649401 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.649665 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.649668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.649669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.649860 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.649929 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.650259 (+ 330us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.650398 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 18:02:48.650571 29321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } }
I20260923 18:02:48.651317 29474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.636079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:40847 (local address 127.0.0.1:42030)
0923 18:02:48.639402 (+ 3323us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:48.639420 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:48.639532 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:48.640529 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:48.640532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:48.640706 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:48.640809 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.640816 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.641858 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.641861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.642631 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:48.642642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.643244 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.643247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:48.643317 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.644122 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:48.644136 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:48.645778 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:48.648967 (+ 3189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.648971 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.648973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.649207 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.649501 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:48.649503 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:48.649505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:48.649579 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:48.649992 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:48.649997 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:48.650091 (+ 94us) client_negotiation.cc:769] Sending connection context
0923 18:02:48.651010 (+ 919us) client_negotiation.cc:240] Negotiation successful
0923 18:02:48.651162 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3257}
I20260923 18:02:48.652067 29321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:02:48.652629 29483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } } }
I20260923 18:02:48.652777 29483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:02:48.652904 29483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:02:48.653030 29483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } }
Sep 23 18:02:48 dist-test-slave-vr57 krb5kdc[29240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.189@KRBTEST.COM
I20260923 18:02:48.655110 29370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b8271528e3646168499979c50716dea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26a840a6a8c24346ad14960fb759b954" is_pre_election: true
I20260923 18:02:48.655471 29370 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b8271528e3646168499979c50716dea in term 0.
I20260923 18:02:48.656152 29269 leader_election.cc:302] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26a840a6a8c24346ad14960fb759b954, 2b8271528e3646168499979c50716dea; no voters:
I20260923 18:02:48.656976 29483 leader_election.cc:288] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26a840a6a8c24346ad14960fb759b954 (127.15.139.189:40103), d1dc51bbfd3141b69d49e432ea7da9b8 (127.15.139.188:40847)
I20260923 18:02:48.657141 29483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:02:48.657258 29483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:02:48.657260 29472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8: Bootstrap starting.
I20260923 18:02:48.657369 29483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:48.654744 29328 master_runner.cc:186] Error getting master registration for 127.15.139.190:33899: OK, instance_id { permanent_uuid: "2b8271528e3646168499979c50716dea" instance_seqno: 1790186568141499 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 18:02:48.658658 29483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } }
I20260923 18:02:48.658949 29328 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 18:02:48.659233 29370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b8271528e3646168499979c50716dea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26a840a6a8c24346ad14960fb759b954"
I20260923 18:02:48.659364 29370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:48.660239 29472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:48.661134 29472 log.cc:824] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:48.661221 29370 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b8271528e3646168499979c50716dea in term 1.
I20260923 18:02:48.661549 29269 leader_election.cc:302] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26a840a6a8c24346ad14960fb759b954, 2b8271528e3646168499979c50716dea; no voters:
I20260923 18:02:48.661625 29483 leader_election.cc:288] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [CANDIDATE]: Term 1 election: Requested vote from peers 26a840a6a8c24346ad14960fb759b954 (127.15.139.189:40103), d1dc51bbfd3141b69d49e432ea7da9b8 (127.15.139.188:40847)
I20260923 18:02:48.661734 29483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:02:48.661818 29483 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [term 1 LEADER]: Becoming Leader. State: Replica: 2b8271528e3646168499979c50716dea, State: Running, Role: LEADER
I20260923 18:02:48.662127 29483 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } }
I20260923 18:02:48.663729 29483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b8271528e3646168499979c50716dea. Latest consensus state: current_term: 1 leader_uuid: "2b8271528e3646168499979c50716dea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } } }
I20260923 18:02:48.663900 29483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [sys.catalog]: This master's current role is: LEADER
W20260923 18:02:48.664777 29493 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:02:48.664893 29493 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:02:48.666782 29472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8: No bootstrap required, opened a new log
W20260923 18:02:48.669412 29505 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:02:48.669559 29505 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:02:48.666226 29253 master_runner.cc:186] Error getting master registration for 127.15.139.188:40847: OK, instance_id { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" instance_seqno: 1790186568593874 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 18:02:48.670667 29253 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 18:02:48.670776 29509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:02:48.671379 29472 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } }
I20260923 18:02:48.671742 29509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:02:48.671860 29472 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:48.671988 29472 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1dc51bbfd3141b69d49e432ea7da9b8, State: Initialized, Role: FOLLOWER
I20260923 18:02:48.672578 29472 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } }
I20260923 18:02:48.674053 29472 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:02:48.675622 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b8271528e3646168499979c50716dea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8"
I20260923 18:02:48.675868 29445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:48.677314 29445 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b8271528e3646168499979c50716dea in term 1.
I20260923 18:02:48.677567 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b8271528e3646168499979c50716dea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" is_pre_election: true
I20260923 18:02:48.677731 29444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [term 1 FOLLOWER]: Refusing update from remote peer 2b8271528e3646168499979c50716dea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:02:48.678052 29511 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } } }
I20260923 18:02:48.678212 29511 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:02:48.678886 29483 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:02:48.664927 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.640140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48906 (local address 127.15.139.189:40103)
0923 18:02:48.640244 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.640248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.644676 (+ 4428us) server_negotiation.cc:407] Connection header received
0923 18:02:48.644801 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.644804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.644878 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.644961 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.645923 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.646887 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.648251 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.648599 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.648679 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.659717 (+ 11038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.659721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.659724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.659755 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.662099 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.662958 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.662961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.662963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.663054 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.663593 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.663599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.663602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.663936 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.664096 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.664642 (+ 546us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.664781 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 18:02:48.677753 29370 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [term 1 FOLLOWER]: Refusing update from remote peer 2b8271528e3646168499979c50716dea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:02:48.681125 29494 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [LEADER]: Connected to new peer: Peer: permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:02:48.683744 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:02:48.684186 29511 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b8271528e3646168499979c50716dea. Latest consensus state: current_term: 1 leader_uuid: "2b8271528e3646168499979c50716dea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } } }
I20260923 18:02:48.685977 29511 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:02:48.687415 29524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b8271528e3646168499979c50716dea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } } }
I20260923 18:02:48.687676 29524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:02:48.687832 29524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b8271528e3646168499979c50716dea. Latest consensus state: current_term: 1 leader_uuid: "2b8271528e3646168499979c50716dea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } } }
I20260923 18:02:48.687994 29524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:02:48.689292 29483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b8271528e3646168499979c50716dea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } } }
I20260923 18:02:48.689417 29483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:48.690591 29511 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b8271528e3646168499979c50716dea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } } }
I20260923 18:02:48.690768 29511 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:02:48.690969 29512 mvcc.cc:205] Tried to move back new op lower bound from 7332604185295261696 to 7332604185242812416. Current Snapshot: MvccSnapshot[applied={T|T < 7332604185295261696}]
I20260923 18:02:48.691805 29525 mvcc.cc:205] Tried to move back new op lower bound from 7332604185295261696 to 7332604185242812416. Current Snapshot: MvccSnapshot[applied={T|T < 7332604185295261696}]
I20260923 18:02:48.693243 29483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b8271528e3646168499979c50716dea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } } }
I20260923 18:02:48.693378 29483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:48.694190 29509 catalog_manager.cc:1347] Generated new cluster ID: a52974b68cd34469b758930795414c81
I20260923 18:02:48.694252 29509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:02:48.694474 29481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b8271528e3646168499979c50716dea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8271528e3646168499979c50716dea" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 33899 } } peers { permanent_uuid: "26a840a6a8c24346ad14960fb759b954" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 40103 } } peers { permanent_uuid: "d1dc51bbfd3141b69d49e432ea7da9b8" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 40847 } } }
I20260923 18:02:48.694583 29481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:02:48.707265 29523 catalog_manager.cc:1259] Loaded cluster ID: a52974b68cd34469b758930795414c81
I20260923 18:02:48.707391 29523 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8: loading cluster ID for follower catalog manager: success
W20260923 18:02:48.707998 29523 catalog_manager.cc:1576] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260923 18:02:48.708163 29523 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:02:48.711314 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.696531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48920 (local address 127.15.139.189:40103)
0923 18:02:48.696643 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.696647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.700433 (+ 3786us) server_negotiation.cc:407] Connection header received
0923 18:02:48.700593 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.700596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.700641 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.700688 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.701664 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.702448 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.703694 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.704050 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.704123 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.706075 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.706079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.706082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.706109 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.707957 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.708449 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.708453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.708455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.708544 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.708839 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.708842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.708844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.709070 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.709167 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.711037 (+ 1870us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.711168 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 18:02:48.714355 29509 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:02:48.716529 29509 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:02:48.718068 29325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.695906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49626 (local address 127.15.139.190:33899)
0923 18:02:48.696021 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.696026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.700123 (+ 4097us) server_negotiation.cc:407] Connection header received
0923 18:02:48.700238 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.700243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.700300 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.700377 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.708290 (+ 7913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.709016 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.710199 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.710605 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.710695 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.714257 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.714260 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.714263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.714292 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.715996 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.716696 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.716702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.716707 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.716803 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.717130 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.717134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.717135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.717332 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.717459 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.717796 (+ 337us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.717926 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 18:02:48.725595 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:48.695919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.15.139.188:40847)
0923 18:02:48.703669 (+ 7750us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:48.703676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:48.703708 (+ 32us) server_negotiation.cc:407] Connection header received
0923 18:02:48.703758 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:48.703761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:48.703812 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:48.703909 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:48.706827 (+ 2918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.707568 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.708664 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:48.709011 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:48.709084 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:48.715609 (+ 6525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:48.715614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:48.715617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:48.715646 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:48.717467 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.724129 (+ 6662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.724133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.724135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.724226 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:48.724514 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:48.724517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:48.724519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:48.724726 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:48.724832 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:48.725273 (+ 441us) server_negotiation.cc:299] Negotiation successful
0923 18:02:48.725401 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7655}
I20260923 18:02:48.727941 29509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b8271528e3646168499979c50716dea: Generated new TSK 0
I20260923 18:02:48.728807 29509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:02:49.682324 29493 catalog_manager.cc:1259] Loaded cluster ID: a52974b68cd34469b758930795414c81
I20260923 18:02:49.682413 29493 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954: loading cluster ID for follower catalog manager: success
I20260923 18:02:49.684255 29493 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954: acquiring CA information for follower catalog manager: success
I20260923 18:02:49.685508 29493 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 26a840a6a8c24346ad14960fb759b954: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 18:02:49.710686 29523 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8: acquiring CA information for follower catalog manager: success
I20260923 18:02:49.712071 29523 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d1dc51bbfd3141b69d49e432ea7da9b8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 18:02:49.799795 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 29253
I20260923 18:02:49.806670 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 29328
I20260923 18:02:49.812594 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 29404
2026-09-23T18:02:49Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3079 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:02:49 dist-test-slave-vr57 krb5kdc[29548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:02:49 dist-test-slave-vr57 krb5kdc[29548](info): set up 2 sockets
Sep 23 18:02:49 dist-test-slave-vr57 krb5kdc[29548](info): commencing operation
krb5kdc: starting...
W20260923 18:02:50.941004 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:02:51 dist-test-slave-vr57 krb5kdc[29548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:02:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:02:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:02:51.066363 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:35923
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:35923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:51.180632 29561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:51.180953 29561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:51.181031 29561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:51.185134 29561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:02:51.185233 29561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:51.185284 29561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:51.185343 29561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:02:51.185384 29561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:02:51.190689 29561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:35923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:35923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:51.191813 29561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:51.192865 29561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:51.201505 29566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:51.201542 29561 server_base.cc:1034] running on GCE node
W20260923 18:02:51.201489 29569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:51.201488 29567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:51.202343 29561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:51.203454 29561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:51.204692 29561 hybrid_clock.cc:648] HybridClock initialized: now 1790186571204658 us; error 49 us; skew 500 ppm
Sep 23 18:02:51 dist-test-slave-vr57 krb5kdc[29548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186571, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:51.207454 29561 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:02:51.208485 29561 webserver.cc:460] Webserver started at http://127.15.139.190:35633/ using document root <none> and password file <none>
I20260923 18:02:51.209003 29561 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:51.209060 29561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:51.209223 29561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:02:51.211227 29561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "54d5c91a53c3448b95a5302d0a9a2af4"
format_stamp: "Formatted at 2026-09-23 18:02:51 on dist-test-slave-vr57"
server_key: "08a3ce871d8d3b664417b1edf2fa9cd5"
server_key_iv: "c23b819c8be55f3fd9ccb046db3b8020"
server_key_version: "encryptionkey@0"
I20260923 18:02:51.211836 29561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "54d5c91a53c3448b95a5302d0a9a2af4"
format_stamp: "Formatted at 2026-09-23 18:02:51 on dist-test-slave-vr57"
server_key: "08a3ce871d8d3b664417b1edf2fa9cd5"
server_key_iv: "c23b819c8be55f3fd9ccb046db3b8020"
server_key_version: "encryptionkey@0"
I20260923 18:02:51.216107 29561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260923 18:02:51.218624 29576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:51.220120 29561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:02:51.220240 29561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "54d5c91a53c3448b95a5302d0a9a2af4"
format_stamp: "Formatted at 2026-09-23 18:02:51 on dist-test-slave-vr57"
server_key: "08a3ce871d8d3b664417b1edf2fa9cd5"
server_key_iv: "c23b819c8be55f3fd9ccb046db3b8020"
server_key_version: "encryptionkey@0"
I20260923 18:02:51.220373 29561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:51.256233 29561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:51.259552 29561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:51.259763 29561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:51.267086 29561 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:35923
I20260923 18:02:51.267113 29628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:35923 every 8 connection(s)
I20260923 18:02:51.268049 29561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:02:51.270932 29629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:51.273260 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 29561
I20260923 18:02:51.273361 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:02:51.273656 15918 external_mini_cluster.cc:1428] Setting key 2289e4ad37a7114c6e3d9bc7d8d0b6ff
I20260923 18:02:51.278419 29629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4: Bootstrap starting.
I20260923 18:02:51.281293 29629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:51.282250 29629 log.cc:824] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4: Log is configured to *not* fsync() on all Append() calls
Sep 23 18:02:51 dist-test-slave-vr57 krb5kdc[29548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:51.284546 29629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4: No bootstrap required, opened a new log
I20260923 18:02:51.289847 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:51.275049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.15.139.190:35923)
0923 18:02:51.275483 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:51.275526 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:51.275576 (+ 50us) server_negotiation.cc:407] Connection header received
0923 18:02:51.276386 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:51.276411 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:51.276859 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:51.277293 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:51.278045 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:51.279695 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:51.280583 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:51.281365 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:51.281551 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:51.284381 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:51.284388 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:51.284404 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:51.284436 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:51.286538 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:51.286989 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:51.286997 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:51.287006 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:51.287208 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:51.287487 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:51.287490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:51.287492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:51.288060 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:51.288285 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:51.288471 (+ 186us) server_negotiation.cc:299] Negotiation successful
0923 18:02:51.288790 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":106,"threads_started":1}
I20260923 18:02:51.289810 29629 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d5c91a53c3448b95a5302d0a9a2af4" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35923 } }
I20260923 18:02:51.290110 29629 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:51.290174 29629 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d5c91a53c3448b95a5302d0a9a2af4, State: Initialized, Role: FOLLOWER
I20260923 18:02:51.290737 29629 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d5c91a53c3448b95a5302d0a9a2af4" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35923 } }
I20260923 18:02:51.290915 29629 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:02:51.291061 29629 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:02:51.291193 29629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:51.292445 29629 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d5c91a53c3448b95a5302d0a9a2af4" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35923 } }
I20260923 18:02:51.292866 29629 leader_election.cc:302] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54d5c91a53c3448b95a5302d0a9a2af4; no voters:
I20260923 18:02:51.293022 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:02:51.293278 29629 leader_election.cc:288] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:02:51.293365 29634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:02:51.293565 29634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [term 1 LEADER]: Becoming Leader. State: Replica: 54d5c91a53c3448b95a5302d0a9a2af4, State: Running, Role: LEADER
I20260923 18:02:51.293949 29634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d5c91a53c3448b95a5302d0a9a2af4" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35923 } }
I20260923 18:02:51.294651 29629 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:02:51.295312 29636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d5c91a53c3448b95a5302d0a9a2af4. Latest consensus state: current_term: 1 leader_uuid: "54d5c91a53c3448b95a5302d0a9a2af4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d5c91a53c3448b95a5302d0a9a2af4" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35923 } } }
I20260923 18:02:51.295454 29636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:51.295856 29635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54d5c91a53c3448b95a5302d0a9a2af4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d5c91a53c3448b95a5302d0a9a2af4" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35923 } } }
I20260923 18:02:51.296039 29639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:02:51.296036 29635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4 [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:51.299167 29639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:02:51.304697 29639 catalog_manager.cc:1347] Generated new cluster ID: 43e01eb602304bbf8309d406c289f6ca
I20260923 18:02:51.304776 29639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:02:51.321313 29639 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:02:51.322667 29639 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:02:51.329625 29639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54d5c91a53c3448b95a5302d0a9a2af4: Generated new TSK 0
I20260923 18:02:51.330431 29639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:02:51.348804 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 29561
2026-09-23T18:02:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1534 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:02:51 dist-test-slave-vr57 krb5kdc[29656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:02:51 dist-test-slave-vr57 krb5kdc[29656](info): set up 2 sockets
Sep 23 18:02:51 dist-test-slave-vr57 krb5kdc[29656](info): commencing operation
krb5kdc: starting...
W20260923 18:02:52.432165 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:02:52 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:02:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:02:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:02:52.559844 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:44585
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:44585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:52.673950 29669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:52.674278 29669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:52.674324 29669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:52.678565 29669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:02:52.678648 29669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:52.678668 29669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:52.678686 29669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:02:52.678702 29669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:02:52.684741 29669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:44585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:44585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:52.685926 29669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:52.687034 29669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:52.696126 29674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:52.696126 29677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:52.696482 29669 server_base.cc:1034] running on GCE node
W20260923 18:02:52.696143 29675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:52.697115 29669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:52.698328 29669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:52.699612 29669 hybrid_clock.cc:648] HybridClock initialized: now 1790186572699576 us; error 51 us; skew 500 ppm
Sep 23 18:02:52 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186572, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:52.702608 29669 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:02:52.703701 29669 webserver.cc:460] Webserver started at http://127.15.139.190:36605/ using document root <none> and password file <none>
I20260923 18:02:52.704304 29669 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:52.704399 29669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:52.704640 29669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:02:52.706866 29669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "2075dc7b9ae049d582addbf085ac51e9"
format_stamp: "Formatted at 2026-09-23 18:02:52 on dist-test-slave-vr57"
server_key: "4cfabfcba9a2a5703b27371f63a15bfc"
server_key_iv: "cafcacf1f54cd0dafb2df8de37c978a2"
server_key_version: "encryptionkey@0"
I20260923 18:02:52.707491 29669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "2075dc7b9ae049d582addbf085ac51e9"
format_stamp: "Formatted at 2026-09-23 18:02:52 on dist-test-slave-vr57"
server_key: "4cfabfcba9a2a5703b27371f63a15bfc"
server_key_iv: "cafcacf1f54cd0dafb2df8de37c978a2"
server_key_version: "encryptionkey@0"
I20260923 18:02:52.711949 29669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260923 18:02:52.715130 29684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:52.716691 29669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:02:52.716825 29669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "2075dc7b9ae049d582addbf085ac51e9"
format_stamp: "Formatted at 2026-09-23 18:02:52 on dist-test-slave-vr57"
server_key: "4cfabfcba9a2a5703b27371f63a15bfc"
server_key_iv: "cafcacf1f54cd0dafb2df8de37c978a2"
server_key_version: "encryptionkey@0"
I20260923 18:02:52.716944 29669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:52.728927 29669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:52.731838 29669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:52.731997 29669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:52.739308 29669 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:44585
I20260923 18:02:52.739670 29736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:44585 every 8 connection(s)
I20260923 18:02:52.740515 29669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:02:52.743806 29737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:52.746716 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 29669
I20260923 18:02:52.746888 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:02:52.747247 15918 external_mini_cluster.cc:1428] Setting key 66d095e183888f5a110d1d35498b71d6
I20260923 18:02:52.750963 29737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Bootstrap starting.
I20260923 18:02:52.753788 29737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:52.754590 29737 log.cc:824] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Log is configured to *not* fsync() on all Append() calls
Sep 23 18:02:52 dist-test-slave-vr57 krb5kdc[29656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:52.756856 29737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: No bootstrap required, opened a new log
I20260923 18:02:52.760568 29737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } }
I20260923 18:02:52.760845 29737 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:52.760910 29737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2075dc7b9ae049d582addbf085ac51e9, State: Initialized, Role: FOLLOWER
I20260923 18:02:52.761435 29737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } }
I20260923 18:02:52.761607 29737 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:02:52.761678 29737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:02:52.761793 29737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:52.762566 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:52.748875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51708 (local address 127.15.139.190:44585)
0923 18:02:52.749404 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:52.749420 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:52.749458 (+ 38us) server_negotiation.cc:407] Connection header received
0923 18:02:52.750127 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:52.750145 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:52.750504 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:52.750942 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:52.751848 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:52.752892 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:52.753812 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:52.754374 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:52.754550 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:52.757353 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:52.757358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:52.757373 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:52.757404 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:52.759209 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:52.759755 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:52.759761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:52.759766 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:52.759891 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:52.760194 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:52.760203 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:52.760204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:52.760626 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:52.760851 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:52.761169 (+ 318us) server_negotiation.cc:299] Negotiation successful
0923 18:02:52.761493 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":170,"threads_started":1}
I20260923 18:02:52.763032 29737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } }
I20260923 18:02:52.763383 29737 leader_election.cc:302] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2075dc7b9ae049d582addbf085ac51e9; no voters:
I20260923 18:02:52.763828 29737 leader_election.cc:288] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:02:52.763924 29742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:02:52.764178 29742 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 1 LEADER]: Becoming Leader. State: Replica: 2075dc7b9ae049d582addbf085ac51e9, State: Running, Role: LEADER
I20260923 18:02:52.764540 29742 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } }
I20260923 18:02:52.765173 29737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:02:52.766381 29744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2075dc7b9ae049d582addbf085ac51e9. Latest consensus state: current_term: 1 leader_uuid: "2075dc7b9ae049d582addbf085ac51e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } } }
I20260923 18:02:52.766531 29744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:52.766856 29743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2075dc7b9ae049d582addbf085ac51e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } } }
I20260923 18:02:52.767006 29743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:52.767813 29751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:02:52.770519 29751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:02:52.776530 29751 catalog_manager.cc:1347] Generated new cluster ID: d341bc3404124bcdbdedf713972d3d66
I20260923 18:02:52.776803 29751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:02:52.802830 29751 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:02:52.803982 29751 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:02:52.818078 29751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Generated new TSK 0
I20260923 18:02:52.818948 29751 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:02:52.859293 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:44585
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:52.984481 29763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:52.984957 29763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:52.985131 29763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:52.989868 29763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:52.989950 29763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:52.990016 29763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:02:52.995852 29763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:44585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:52.997049 29763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:52.998121 29763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:53.008291 29771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:53.008381 29768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:53.008467 29763 server_base.cc:1034] running on GCE node
W20260923 18:02:53.008739 29769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:53.009238 29763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:53.009958 29763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:53.011157 29763 hybrid_clock.cc:648] HybridClock initialized: now 1790186573011131 us; error 41 us; skew 500 ppm
Sep 23 18:02:53 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:53.014479 29763 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:02:53.015627 29763 webserver.cc:460] Webserver started at http://127.15.139.129:40997/ using document root <none> and password file <none>
I20260923 18:02:53.016275 29763 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:53.016377 29763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:53.016620 29763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:02:53.018870 29763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "6c301bdb7c55451e96793f78aec269ae"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "92b2c59d391aa99f3e322de2aa4e5123"
server_key_iv: "97841cda20eb4b8056f98687409dc7f3"
server_key_version: "encryptionkey@0"
I20260923 18:02:53.019484 29763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "6c301bdb7c55451e96793f78aec269ae"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "92b2c59d391aa99f3e322de2aa4e5123"
server_key_iv: "97841cda20eb4b8056f98687409dc7f3"
server_key_version: "encryptionkey@0"
I20260923 18:02:53.023837 29763 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260923 18:02:53.026700 29778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:53.028231 29763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:02:53.028380 29763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "6c301bdb7c55451e96793f78aec269ae"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "92b2c59d391aa99f3e322de2aa4e5123"
server_key_iv: "97841cda20eb4b8056f98687409dc7f3"
server_key_version: "encryptionkey@0"
I20260923 18:02:53.028609 29763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:53.049938 29763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:53.053728 29763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:53.053962 29763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:53.054621 29763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:53.055601 29763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:02:53.055667 29763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:53.055751 29763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:02:53.055799 29763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:53.065126 29763 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:36979
I20260923 18:02:53.065148 29891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:36979 every 8 connection(s)
I20260923 18:02:53.066108 29763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:02:53.067672 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 29763
I20260923 18:02:53.067819 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:02:53.068179 15918 external_mini_cluster.cc:1428] Setting key b898efb7133083b5141807c880647b09
Sep 23 18:02:53 dist-test-slave-vr57 krb5kdc[29656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:02:53.089545 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.068182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:32793 (local address 127.15.139.190:44585)
0923 18:02:53.068349 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:53.068354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:53.069390 (+ 1036us) server_negotiation.cc:407] Connection header received
0923 18:02:53.070845 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:53.070851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:53.070916 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:53.071018 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:53.072878 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.073432 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.075405 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.075731 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.075822 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.078864 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:53.078867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:53.078869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:53.078892 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:53.087200 (+ 8308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:53.087904 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:53.087909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:53.087910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:53.088000 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:53.088379 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:53.088383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:53.088385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:53.088593 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:53.088709 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:53.089237 (+ 528us) server_negotiation.cc:299] Negotiation successful
0923 18:02:53.089378 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 18:02:53.090618 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.068652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:32793)
0923 18:02:53.069151 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:53.069204 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:53.070608 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:53.071351 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:53.071364 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:53.071944 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:53.072702 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.072727 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.073576 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.073582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.075252 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.075268 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.075914 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.075921 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.076155 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.076822 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:53.076848 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:53.078677 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:53.087356 (+ 8679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:53.087363 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:53.087382 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:53.087783 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:53.088103 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:53.088106 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:53.088108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:53.088278 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:53.088702 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:53.088709 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:53.088995 (+ 286us) client_negotiation.cc:769] Sending connection context
0923 18:02:53.089196 (+ 201us) client_negotiation.cc:240] Negotiation successful
0923 18:02:53.089608 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":136,"threads_started":1}
I20260923 18:02:53.092103 29892 heartbeater.cc:347] Connected to a master server at 127.15.139.190:44585
I20260923 18:02:53.092381 29892 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:53.092941 29892 heartbeater.cc:511] Master 127.15.139.190:44585 requested a full tablet report, sending...
I20260923 18:02:53.094766 29701 ts_manager.cc:194] Registered new tserver with Master: 6c301bdb7c55451e96793f78aec269ae (127.15.139.129:36979)
I20260923 18:02:53.096108 29701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:32793
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:02:53.098766 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:44585
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:53.210886 29897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:53.211182 29897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:53.211261 29897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:53.215380 29897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:53.215468 29897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:53.215577 29897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:02:53.220885 29897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:44585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:53.221952 29897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:53.223027 29897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:53.232857 29905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:53.232873 29903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:53.232887 29902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:53.233183 29897 server_base.cc:1034] running on GCE node
I20260923 18:02:53.233695 29897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:53.234383 29897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:53.235603 29897 hybrid_clock.cc:648] HybridClock initialized: now 1790186573235565 us; error 47 us; skew 500 ppm
Sep 23 18:02:53 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:53.238704 29897 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:02:53.239748 29897 webserver.cc:460] Webserver started at http://127.15.139.130:43917/ using document root <none> and password file <none>
I20260923 18:02:53.240343 29897 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:53.240410 29897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:53.240649 29897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:02:53.242852 29897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "a1f901ffff9e44949af805dd382c630f"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "d5caf35596f566cca7182e31792477b7"
server_key_iv: "e7b0041ce10e878c703fa59310a47887"
server_key_version: "encryptionkey@0"
I20260923 18:02:53.243458 29897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "a1f901ffff9e44949af805dd382c630f"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "d5caf35596f566cca7182e31792477b7"
server_key_iv: "e7b0041ce10e878c703fa59310a47887"
server_key_version: "encryptionkey@0"
I20260923 18:02:53.247653 29897 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:02:53.250201 29912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:53.251609 29897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:02:53.251748 29897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "a1f901ffff9e44949af805dd382c630f"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "d5caf35596f566cca7182e31792477b7"
server_key_iv: "e7b0041ce10e878c703fa59310a47887"
server_key_version: "encryptionkey@0"
I20260923 18:02:53.251866 29897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:53.288430 29897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:53.291867 29897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:53.292073 29897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:53.292668 29897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:53.293505 29897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:02:53.293576 29897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:53.293648 29897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:02:53.293694 29897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:53.302223 29897 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:40369
I20260923 18:02:53.302250 30025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:40369 every 8 connection(s)
I20260923 18:02:53.303463 29897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:02:53.305964 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 29897
I20260923 18:02:53.306093 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:02:53.306403 15918 external_mini_cluster.cc:1428] Setting key ffe0d97fbcdf4ce68d32041b530e5d9d
Sep 23 18:02:53 dist-test-slave-vr57 krb5kdc[29656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
I20260923 18:02:53.318807 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.305451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:51091 (local address 127.15.139.190:44585)
0923 18:02:53.305589 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:53.305593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:53.306683 (+ 1090us) server_negotiation.cc:407] Connection header received
0923 18:02:53.308396 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:53.308400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:53.308454 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:53.308535 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:53.309766 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.310304 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.311476 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.311818 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.311937 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.314692 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:53.314695 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:53.314698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:53.314733 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:53.316299 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:53.316971 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:53.316974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:53.316976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:53.317060 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:53.317531 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:53.317534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:53.317535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:53.317810 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:53.317908 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:53.318557 (+ 649us) server_negotiation.cc:299] Negotiation successful
0923 18:02:53.318671 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 18:02:53.320107 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.305893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:51091)
0923 18:02:53.306509 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:53.306569 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:53.308171 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:53.308715 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:53.308723 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:53.309094 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:53.309606 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.309620 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.310436 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.310440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.311341 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.311351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.311981 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.311985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.312154 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.312782 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:53.312798 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:53.314513 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:53.316435 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:53.316441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:53.316459 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:53.316854 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:53.317191 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:53.317194 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:53.317196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:53.317393 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:53.317921 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:53.317927 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:53.318245 (+ 318us) client_negotiation.cc:769] Sending connection context
0923 18:02:53.318478 (+ 233us) client_negotiation.cc:240] Negotiation successful
0923 18:02:53.318870 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":147,"threads_started":1}
I20260923 18:02:53.321305 30026 heartbeater.cc:347] Connected to a master server at 127.15.139.190:44585
I20260923 18:02:53.321557 30026 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:53.322139 30026 heartbeater.cc:511] Master 127.15.139.190:44585 requested a full tablet report, sending...
I20260923 18:02:53.323346 29701 ts_manager.cc:194] Registered new tserver with Master: a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369)
I20260923 18:02:53.324082 29701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:51091
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:02:53.334851 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:44585
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:53.446446 30031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:53.446707 30031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:53.446749 30031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:53.450717 30031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:53.450778 30031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:53.450834 30031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:02:53.456030 30031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:44585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:53.457119 30031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:53.458144 30031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:53.467898 30037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:53.467898 30036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:53.467916 30039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:53.468262 30031 server_base.cc:1034] running on GCE node
I20260923 18:02:53.468693 30031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:53.469301 30031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:53.470440 30031 hybrid_clock.cc:648] HybridClock initialized: now 1790186573470417 us; error 36 us; skew 500 ppm
Sep 23 18:02:53 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:53.473310 30031 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:02:53.474160 30031 webserver.cc:460] Webserver started at http://127.15.139.131:38927/ using document root <none> and password file <none>
I20260923 18:02:53.474715 30031 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:53.474776 30031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:53.475035 30031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:02:53.477126 30031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "8975ed20d415485ba322c4d9be7ef71d"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "cc21f2117362e23cfecaabf8c063adfa"
server_key_iv: "f2aad0cd4bed0d2088a95f2ac94b53cb"
server_key_version: "encryptionkey@0"
I20260923 18:02:53.477685 30031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "8975ed20d415485ba322c4d9be7ef71d"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "cc21f2117362e23cfecaabf8c063adfa"
server_key_iv: "f2aad0cd4bed0d2088a95f2ac94b53cb"
server_key_version: "encryptionkey@0"
I20260923 18:02:53.481467 30031 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260923 18:02:53.484090 30046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:53.485741 30031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:02:53.485910 30031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "8975ed20d415485ba322c4d9be7ef71d"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "cc21f2117362e23cfecaabf8c063adfa"
server_key_iv: "f2aad0cd4bed0d2088a95f2ac94b53cb"
server_key_version: "encryptionkey@0"
I20260923 18:02:53.486032 30031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:53.504601 30031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:53.508193 30031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:53.508410 30031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:53.509048 30031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:53.509912 30031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:02:53.509989 30031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:53.510083 30031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:02:53.510135 30031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:53.519330 30031 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:35579
I20260923 18:02:53.519383 30159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:35579 every 8 connection(s)
I20260923 18:02:53.520376 30031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:02:53.521349 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 30031
I20260923 18:02:53.521461 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:02:53.521763 15918 external_mini_cluster.cc:1428] Setting key e60bd83b5948c816d4e081d2ea4987d0
Sep 23 18:02:53 dist-test-slave-vr57 krb5kdc[29656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:53.535987 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.522531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:41431 (local address 127.15.139.190:44585)
0923 18:02:53.522723 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:53.522727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:53.524015 (+ 1288us) server_negotiation.cc:407] Connection header received
0923 18:02:53.525558 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:53.525562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:53.525623 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:53.525713 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:53.526994 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.527625 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.528853 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.529096 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.529239 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.531928 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:53.531933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:53.531936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:53.531971 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:53.533490 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:53.534058 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:53.534064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:53.534066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:53.534159 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:53.534600 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:53.534602 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:53.534604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:53.534878 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:53.534967 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:53.535711 (+ 744us) server_negotiation.cc:299] Negotiation successful
0923 18:02:53.535840 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 18:02:53.537009 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.523165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:41431)
0923 18:02:53.523832 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:53.523921 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:53.525340 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:53.525887 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:53.525895 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:53.526302 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:53.526820 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.526836 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.527780 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.527783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.528710 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.528720 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.529282 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.529286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.529473 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.530043 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:53.530061 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:53.531748 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:53.533619 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:53.533627 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:53.533643 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:53.533941 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:53.534291 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:53.534295 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:53.534297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:53.534484 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:53.535037 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:53.535046 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:53.535397 (+ 351us) client_negotiation.cc:769] Sending connection context
0923 18:02:53.535643 (+ 246us) client_negotiation.cc:240] Negotiation successful
0923 18:02:53.536001 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":161,"threads_started":1}
I20260923 18:02:53.538269 30160 heartbeater.cc:347] Connected to a master server at 127.15.139.190:44585
I20260923 18:02:53.538496 30160 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:53.538946 30160 heartbeater.cc:511] Master 127.15.139.190:44585 requested a full tablet report, sending...
I20260923 18:02:53.539966 29701 ts_manager.cc:194] Registered new tserver with Master: 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579)
I20260923 18:02:53.540526 29701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:41431
I20260923 18:02:53.548583 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:02:53.558441 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.550658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51718 (local address 127.15.139.190:44585)
0923 18:02:53.550808 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:53.550813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:53.551023 (+ 210us) server_negotiation.cc:407] Connection header received
0923 18:02:53.551125 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:53.551127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:53.551168 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:53.551234 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:53.552124 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.552644 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.553452 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.553652 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.553728 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.555132 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:53.555135 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:53.555137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:53.555160 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:53.556694 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:53.557139 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:53.557141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:53.557143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:53.557241 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:53.557557 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:53.557559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:53.557560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:53.557806 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:53.557910 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:53.558183 (+ 273us) server_negotiation.cc:299] Negotiation successful
0923 18:02:53.558291 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 18:02:53.561560 29701 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:02:53.564575 29701 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:02:53.579646 30171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.573776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:36979 (local address 127.0.0.1:38202)
0923 18:02:53.574105 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:53.574152 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:53.574442 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:53.575534 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:53.575540 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:53.575564 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:53.575707 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.575714 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.576949 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.576956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.578203 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.578213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.579047 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.579050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.579167 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.579203 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 18:02:53.579341 (+ 138us) client_negotiation.cc:240] Negotiation successful
0923 18:02:53.579467 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":58,"thread_start_us":98,"threads_started":1}
I20260923 18:02:53.579648 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.573926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38202 (local address 127.15.139.129:36979)
0923 18:02:53.574281 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:53.574309 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:53.574333 (+ 24us) server_negotiation.cc:407] Connection header received
0923 18:02:53.574666 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:53.574673 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:53.574905 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:53.575071 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:53.575836 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.576820 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.578330 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.578920 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.579063 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.579207 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:53.579379 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 18:02:53.579476 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":160,"threads_started":1}
I20260923 18:02:53.580564 30173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.574504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:35579 (local address 127.0.0.1:58784)
0923 18:02:53.575265 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:53.575282 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:53.575406 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:53.575963 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:53.575965 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:53.575984 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:53.576081 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.576087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.577547 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.577554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.578933 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.578946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.580156 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.580159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.580259 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.580282 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 18:02:53.580364 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 18:02:53.580435 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"mutex_wait_us":64,"thread_start_us":91,"threads_started":1}
I20260923 18:02:53.580699 30175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.574801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58784 (local address 127.15.139.131:35579)
0923 18:02:53.575288 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:53.575302 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:53.575367 (+ 65us) server_negotiation.cc:407] Connection header received
0923 18:02:53.575609 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:53.575616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:53.575814 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:53.575985 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:53.576195 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.577369 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.579098 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.580002 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.580136 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.580265 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:53.580407 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 18:02:53.580508 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":160,"threads_started":1}
I20260923 18:02:53.580845 30174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.574816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:40369 (local address 127.0.0.1:53436)
0923 18:02:53.575277 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:53.575291 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:53.575380 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:53.576197 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:53.576200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:53.576215 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:53.576294 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.576299 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.577648 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.577651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.578934 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.578946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.580369 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.580375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.580489 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.580515 (+ 26us) client_negotiation.cc:769] Sending connection context
0923 18:02:53.580596 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 18:02:53.580664 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":102,"thread_start_us":109,"threads_started":1}
I20260923 18:02:53.581290 30176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.575264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.15.139.130:40369)
0923 18:02:53.575630 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:53.575644 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:53.575667 (+ 23us) server_negotiation.cc:407] Connection header received
0923 18:02:53.575748 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:53.575752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:53.575908 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:53.576048 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:53.576420 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.577496 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.579098 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.580002 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.580136 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.580265 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:53.581033 (+ 768us) server_negotiation.cc:299] Negotiation successful
0923 18:02:53.581135 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":132,"threads_started":1}
I20260923 18:02:53.583366 29826 tablet_service.cc:1467] Processing CreateTablet for tablet 647415fa3d8d444cadedaa4a31e7e8e5 (DEFAULT_TABLE table=test-table [id=430a335964a14ee1b5a818884fa69006]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:02:53.584510 29826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647415fa3d8d444cadedaa4a31e7e8e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:53.584204 29959 tablet_service.cc:1467] Processing CreateTablet for tablet 647415fa3d8d444cadedaa4a31e7e8e5 (DEFAULT_TABLE table=test-table [id=430a335964a14ee1b5a818884fa69006]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:02:53.585306 29959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647415fa3d8d444cadedaa4a31e7e8e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:53.585538 30094 tablet_service.cc:1467] Processing CreateTablet for tablet 647415fa3d8d444cadedaa4a31e7e8e5 (DEFAULT_TABLE table=test-table [id=430a335964a14ee1b5a818884fa69006]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:02:53.586516 30094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647415fa3d8d444cadedaa4a31e7e8e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:53.591641 30177 tablet_bootstrap.cc:492] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Bootstrap starting.
I20260923 18:02:53.593129 30178 tablet_bootstrap.cc:492] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Bootstrap starting.
I20260923 18:02:53.593713 30179 tablet_bootstrap.cc:492] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d: Bootstrap starting.
I20260923 18:02:53.594000 30177 tablet_bootstrap.cc:654] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:53.594938 30177 log.cc:824] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:53.595978 30178 tablet_bootstrap.cc:654] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:53.595988 30179 tablet_bootstrap.cc:654] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:53.596647 30178 log.cc:824] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:53.597162 30179 log.cc:824] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:53.597213 30177 tablet_bootstrap.cc:492] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: No bootstrap required, opened a new log
I20260923 18:02:53.597406 30177 ts_tablet_manager.cc:1397] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260923 18:02:53.598367 30178 tablet_bootstrap.cc:492] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: No bootstrap required, opened a new log
I20260923 18:02:53.598552 30178 ts_tablet_manager.cc:1397] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260923 18:02:53.601406 30179 tablet_bootstrap.cc:492] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d: No bootstrap required, opened a new log
I20260923 18:02:53.601305 30177 raft_consensus.cc:348] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:53.601559 30177 raft_consensus.cc:374] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:53.601601 30179 ts_tablet_manager.cc:1397] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260923 18:02:53.601611 30177 raft_consensus.cc:729] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c301bdb7c55451e96793f78aec269ae, State: Initialized, Role: FOLLOWER
I20260923 18:02:53.602061 30177 consensus_queue.cc:260] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:53.602038 30178 raft_consensus.cc:348] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:53.602283 30178 raft_consensus.cc:374] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:53.602331 30178 raft_consensus.cc:729] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1f901ffff9e44949af805dd382c630f, State: Initialized, Role: FOLLOWER
I20260923 18:02:53.602736 30178 consensus_queue.cc:260] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:53.603075 30177 ts_tablet_manager.cc:1428] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260923 18:02:53.603426 30026 heartbeater.cc:503] Master 127.15.139.190:44585 was elected leader, sending a full tablet report...
I20260923 18:02:53.603425 29892 heartbeater.cc:503] Master 127.15.139.190:44585 was elected leader, sending a full tablet report...
I20260923 18:02:53.603658 30178 ts_tablet_manager.cc:1428] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260923 18:02:53.605466 30179 raft_consensus.cc:348] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:53.605717 30179 raft_consensus.cc:374] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:53.605795 30179 raft_consensus.cc:729] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8975ed20d415485ba322c4d9be7ef71d, State: Initialized, Role: FOLLOWER
I20260923 18:02:53.606204 30179 consensus_queue.cc:260] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:53.606787 30160 heartbeater.cc:503] Master 127.15.139.190:44585 was elected leader, sending a full tablet report...
I20260923 18:02:53.607126 30179 ts_tablet_manager.cc:1428] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260923 18:02:53.748029 30184 raft_consensus.cc:482] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:02:53.748260 30184 raft_consensus.cc:504] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:53.749665 30184 leader_election.cc:288] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c301bdb7c55451e96793f78aec269ae (127.15.139.129:36979), 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579)
I20260923 18:02:53.754395 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.750150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:48687 (local address 127.15.139.129:36979)
0923 18:02:53.750276 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:53.750281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:53.750300 (+ 19us) server_negotiation.cc:407] Connection header received
0923 18:02:53.750533 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:53.750537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:53.750600 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:53.750678 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:53.751117 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.751846 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.753279 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.753864 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.753991 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.754045 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:53.754194 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 18:02:53.754271 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 18:02:53.754465 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.750042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:36979 (local address 127.15.139.130:48687)
0923 18:02:53.750188 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:53.750213 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:53.750323 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:53.750830 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:53.750833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:53.750860 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:53.750998 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.751004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.751956 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.751963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.753138 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.753147 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.753984 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.753986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.754065 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.754085 (+ 20us) client_negotiation.cc:769] Sending connection context
0923 18:02:53.754146 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 18:02:53.754315 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"spinlock_wait_cycles":1792}
I20260923 18:02:53.754716 30175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.750520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:48559 (local address 127.15.139.131:35579)
0923 18:02:53.750628 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:53.750632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:53.750857 (+ 225us) server_negotiation.cc:407] Connection header received
0923 18:02:53.750957 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:53.750960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:53.751005 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:53.751091 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:53.751472 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.752206 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.753500 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.754087 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.754227 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.754270 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:53.754489 (+ 219us) server_negotiation.cc:299] Negotiation successful
0923 18:02:53.754569 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 18:02:53.754751 30186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.750337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:35579 (local address 127.15.139.130:48559)
0923 18:02:53.750751 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:53.750765 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:53.750870 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:53.751143 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:53.751146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:53.751164 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:53.751345 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.751352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.752333 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.752337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.753390 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:53.753396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.754243 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.754246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:53.754355 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.754378 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 18:02:53.754465 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 18:02:53.754542 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"spinlock_wait_cycles":8320,"thread_start_us":116,"threads_started":1}
I20260923 18:02:53.755188 29846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647415fa3d8d444cadedaa4a31e7e8e5" candidate_uuid: "a1f901ffff9e44949af805dd382c630f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c301bdb7c55451e96793f78aec269ae" is_pre_election: true
I20260923 18:02:53.755429 30114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647415fa3d8d444cadedaa4a31e7e8e5" candidate_uuid: "a1f901ffff9e44949af805dd382c630f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8975ed20d415485ba322c4d9be7ef71d" is_pre_election: true
I20260923 18:02:53.755550 29846 raft_consensus.cc:2454] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1f901ffff9e44949af805dd382c630f in term 0.
I20260923 18:02:53.755741 30114 raft_consensus.cc:2454] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1f901ffff9e44949af805dd382c630f in term 0.
I20260923 18:02:53.756122 29914 leader_election.cc:302] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c301bdb7c55451e96793f78aec269ae, a1f901ffff9e44949af805dd382c630f; no voters:
I20260923 18:02:53.756398 30184 raft_consensus.cc:2781] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:02:53.756488 30184 raft_consensus.cc:482] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:02:53.756533 30184 raft_consensus.cc:3037] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:53.757575 30184 raft_consensus.cc:504] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:53.757920 30184 leader_election.cc:288] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [CANDIDATE]: Term 1 election: Requested vote from peers 6c301bdb7c55451e96793f78aec269ae (127.15.139.129:36979), 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579)
I20260923 18:02:53.758304 29846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647415fa3d8d444cadedaa4a31e7e8e5" candidate_uuid: "a1f901ffff9e44949af805dd382c630f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c301bdb7c55451e96793f78aec269ae"
I20260923 18:02:53.758353 30114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647415fa3d8d444cadedaa4a31e7e8e5" candidate_uuid: "a1f901ffff9e44949af805dd382c630f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8975ed20d415485ba322c4d9be7ef71d"
I20260923 18:02:53.758440 29846 raft_consensus.cc:3037] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:53.758456 30114 raft_consensus.cc:3037] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:53.759766 29846 raft_consensus.cc:2454] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1f901ffff9e44949af805dd382c630f in term 1.
I20260923 18:02:53.759766 30114 raft_consensus.cc:2454] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1f901ffff9e44949af805dd382c630f in term 1.
I20260923 18:02:53.760039 29914 leader_election.cc:302] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8975ed20d415485ba322c4d9be7ef71d, a1f901ffff9e44949af805dd382c630f; no voters:
I20260923 18:02:53.760212 30184 raft_consensus.cc:2781] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:02:53.760464 30184 raft_consensus.cc:686] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 1 LEADER]: Becoming Leader. State: Replica: a1f901ffff9e44949af805dd382c630f, State: Running, Role: LEADER
I20260923 18:02:53.760795 30184 consensus_queue.cc:237] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:53.764287 29697 catalog_manager.cc:5495] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f reported cstate change: term changed from 0 to 1, leader changed from <none> to a1f901ffff9e44949af805dd382c630f (127.15.139.130). New cstate: current_term: 1 leader_uuid: "a1f901ffff9e44949af805dd382c630f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } health_report { overall_health: HEALTHY } } }
W20260923 18:02:53.771438 30161 tablet.cc:2369] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:02:53.804773 30027 tablet.cc:2369] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:02:53.817440 29893 tablet.cc:2369] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:02:53.820509 30176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.816801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.15.139.130:40369)
0923 18:02:53.816977 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:53.816982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:53.816997 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:02:53.817076 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:53.817078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:53.817132 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:53.817250 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:02:53.817546 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.818108 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.818988 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:53.819209 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:53.819322 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:53.819683 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:02:53.820104 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:02:53.820265 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:53.820302 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 18:02:53.820359 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260923 18:02:53.826238 30114 raft_consensus.cc:1261] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d [term 1 FOLLOWER]: Refusing update from remote peer a1f901ffff9e44949af805dd382c630f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:02:53.826267 29846 raft_consensus.cc:1261] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Refusing update from remote peer a1f901ffff9e44949af805dd382c630f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:02:53.826979 30184 consensus_queue.cc:1035] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:02:53.827163 30187 consensus_queue.cc:1035] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:02:53.835227 30191 mvcc.cc:205] Tried to move back new op lower bound from 7332604206385324032 to 7332604206127026176. Current Snapshot: MvccSnapshot[applied={T|T < 7332604206385324032}]
I20260923 18:02:53.842998 29697 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:02:53.843147 29697 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:02:53.845772 29697 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Sending DeleteTablet for 3 replicas of tablet 647415fa3d8d444cadedaa4a31e7e8e5
I20260923 18:02:53.846576 29826 tablet_service.cc:1514] Processing DeleteTablet for tablet 647415fa3d8d444cadedaa4a31e7e8e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:02:53 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:38202
I20260923 18:02:53.846719 30094 tablet_service.cc:1514] Processing DeleteTablet for tablet 647415fa3d8d444cadedaa4a31e7e8e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:02:53 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:58784
I20260923 18:02:53.847160 30200 tablet_replica.cc:330] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: stopping tablet replica
I20260923 18:02:53.847299 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 29763
I20260923 18:02:53.847399 29959 tablet_service.cc:1514] Processing DeleteTablet for tablet 647415fa3d8d444cadedaa4a31e7e8e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:02:53 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:53436
I20260923 18:02:53.847464 30201 tablet_replica.cc:330] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d: stopping tablet replica
I20260923 18:02:53.847815 30201 raft_consensus.cc:2229] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:02:53.848134 30201 raft_consensus.cc:2258] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:02:53.850376 30201 ts_tablet_manager.cc:1905] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:02:53.851745 30202 tablet_replica.cc:330] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: stopping tablet replica
I20260923 18:02:53.852126 30202 raft_consensus.cc:2229] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:02:53.852617 30202 raft_consensus.cc:2258] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:02:53.853039 30201 ts_tablet_manager.cc:1918] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:02:53.853137 30201 log.cc:1196] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/647415fa3d8d444cadedaa4a31e7e8e5
I20260923 18:02:53.853520 30201 ts_tablet_manager.cc:1939] T 647415fa3d8d444cadedaa4a31e7e8e5 P 8975ed20d415485ba322c4d9be7ef71d: Deleting consensus metadata
I20260923 18:02:53.854506 30202 ts_tablet_manager.cc:1905] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:02:53.854805 29686 catalog_manager.cc:4841] TS 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579): tablet 647415fa3d8d444cadedaa4a31e7e8e5 (table test-table [id=430a335964a14ee1b5a818884fa69006]) successfully deleted
I20260923 18:02:53.855072 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 29897
W20260923 18:02:53.855129 29686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:36979 (error 108)
I20260923 18:02:53.857439 30174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.857137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:36979 (local address 127.0.0.1:38206)
0923 18:02:53.857270 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:53.857332 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:36979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260923 18:02:53.857626 29686 catalog_manager.cc:4584] TS 6c301bdb7c55451e96793f78aec269ae (127.15.139.129:36979): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 647415fa3d8d444cadedaa4a31e7e8e5: Network error: Client connection negotiation failed: client connection to 127.15.139.129:36979: connect: Connection refused (error 111)
I20260923 18:02:53.861474 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 30031
I20260923 18:02:53.861929 30174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:53.861577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:40369 (local address 127.0.0.1:53446)
0923 18:02:53.861772 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:53.861831 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.130:40369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260923 18:02:53.867266 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 29669
I20260923 18:02:53.873615 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:44585
--webserver_interface=127.15.139.190
--webserver_port=36605
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:44585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:53.986872 30204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:53.987187 30204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:53.987231 30204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:53.991462 30204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:02:53.991549 30204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:53.991569 30204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:53.991587 30204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:02:53.991626 30204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:02:53.997084 30204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:44585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:44585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=36605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:53.998224 30204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:53.999460 30204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:54.012986 30211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:54.012986 30210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:54.012982 30213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:54.013351 30204 server_base.cc:1034] running on GCE node
I20260923 18:02:54.014034 30204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:54.015280 30204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:54.016484 30204 hybrid_clock.cc:648] HybridClock initialized: now 1790186574016442 us; error 58 us; skew 500 ppm
Sep 23 18:02:54 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186574, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:54.019646 30204 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:02:54.020643 30204 webserver.cc:460] Webserver started at http://127.15.139.190:36605/ using document root <none> and password file <none>
I20260923 18:02:54.021286 30204 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:54.021399 30204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:54.025435 30204 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:02:54.027716 30220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:54.028932 30204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:02:54.029075 30204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "2075dc7b9ae049d582addbf085ac51e9"
format_stamp: "Formatted at 2026-09-23 18:02:52 on dist-test-slave-vr57"
server_key: "4cfabfcba9a2a5703b27371f63a15bfc"
server_key_iv: "cafcacf1f54cd0dafb2df8de37c978a2"
server_key_version: "encryptionkey@0"
I20260923 18:02:54.029693 30204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:54.041919 30204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:54.045207 30204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:54.045416 30204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:54.052870 30204 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:44585
I20260923 18:02:54.053107 30272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:44585 every 8 connection(s)
I20260923 18:02:54.054028 30204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:02:54.058637 30273 sys_catalog.cc:263] Verifying existing consensus state
I20260923 18:02:54.060446 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 30204
I20260923 18:02:54.060464 30273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Bootstrap starting.
I20260923 18:02:54.060949 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:36979
--local_ip_for_outbound_sockets=127.15.139.129
--tserver_master_addrs=127.15.139.190:44585
--webserver_port=40997
--webserver_interface=127.15.139.129
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:02:54.078527 30273 log.cc:824] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:54.088546 30273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 18:02:54.088940 30273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Bootstrap complete.
I20260923 18:02:54.093045 30273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } }
I20260923 18:02:54.093433 30273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2075dc7b9ae049d582addbf085ac51e9, State: Initialized, Role: FOLLOWER
I20260923 18:02:54.093941 30273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } }
I20260923 18:02:54.094059 30273 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:02:54.094151 30273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:02:54.094270 30273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 1 FOLLOWER]: Advancing to term 2
I20260923 18:02:54.096514 30273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } }
I20260923 18:02:54.096868 30273 leader_election.cc:302] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2075dc7b9ae049d582addbf085ac51e9; no voters:
I20260923 18:02:54.097191 30273 leader_election.cc:288] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 18:02:54.097283 30277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 18:02:54.097518 30277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 2 LEADER]: Becoming Leader. State: Replica: 2075dc7b9ae049d582addbf085ac51e9, State: Running, Role: LEADER
I20260923 18:02:54.097817 30277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } }
I20260923 18:02:54.098455 30273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:02:54.098734 30277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2075dc7b9ae049d582addbf085ac51e9. Latest consensus state: current_term: 2 leader_uuid: "2075dc7b9ae049d582addbf085ac51e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } } }
I20260923 18:02:54.098852 30277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:54.099192 30281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:02:54.099349 30278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2075dc7b9ae049d582addbf085ac51e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } } }
I20260923 18:02:54.099568 30278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:54.104120 30281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:02:54.104780 30281 catalog_manager.cc:1259] Loaded cluster ID: d341bc3404124bcdbdedf713972d3d66
I20260923 18:02:54.104856 30281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:02:54.106024 30281 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260923 18:02:54.108893 30281 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260923 18:02:54.113148 30281 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:02:54.113775 30281 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Loaded TSK: 0
I20260923 18:02:54.114743 30281 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 18:02:54.187114 30275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:54.187467 30275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:54.187570 30275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:54.191815 30275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:54.191905 30275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:54.191998 30275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:02:54.197427 30275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:36979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=40997
--tserver_master_addrs=127.15.139.190:44585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:54.198585 30275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:54.199839 30275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:54.210440 30300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:54.210628 30301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:54.211040 30303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:54.211198 30275 server_base.cc:1034] running on GCE node
I20260923 18:02:54.211795 30275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:54.212558 30275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:54.213778 30275 hybrid_clock.cc:648] HybridClock initialized: now 1790186574213747 us; error 56 us; skew 500 ppm
Sep 23 18:02:54 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186574, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:54.217017 30275 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:02:54.218076 30275 webserver.cc:460] Webserver started at http://127.15.139.129:40997/ using document root <none> and password file <none>
I20260923 18:02:54.218693 30275 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:54.218777 30275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:54.223115 30275 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:02:54.225620 30310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:54.226999 30275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260923 18:02:54.227139 30275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "6c301bdb7c55451e96793f78aec269ae"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "92b2c59d391aa99f3e322de2aa4e5123"
server_key_iv: "97841cda20eb4b8056f98687409dc7f3"
server_key_version: "encryptionkey@0"
I20260923 18:02:54.227841 30275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:54.253398 30275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:54.256953 30275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:54.257165 30275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:54.257843 30275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:54.259047 30317 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 18:02:54.263790 30275 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 18:02:54.263882 30275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260923 18:02:54.263975 30275 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 18:02:54.265702 30275 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 18:02:54.265753 30275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260923 18:02:54.266093 30317 tablet_bootstrap.cc:492] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Bootstrap starting.
I20260923 18:02:54.276739 30275 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:36979
I20260923 18:02:54.276806 30424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:36979 every 8 connection(s)
I20260923 18:02:54.277724 30275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:02:54.278432 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 30275
I20260923 18:02:54.278853 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:40369
--local_ip_for_outbound_sockets=127.15.139.130
--tserver_master_addrs=127.15.139.190:44585
--webserver_port=43917
--webserver_interface=127.15.139.130
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:02:54.285236 30317 log.cc:824] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:54.293329 30317 tablet_bootstrap.cc:492] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
Sep 23 18:02:54 dist-test-slave-vr57 krb5kdc[29656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186574, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:54.293839 30317 tablet_bootstrap.cc:492] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Bootstrap complete.
I20260923 18:02:54.295177 30317 ts_tablet_manager.cc:1397] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Time spent bootstrapping tablet: real 0.029s user 0.013s sys 0.011s
I20260923 18:02:54.298007 30317 raft_consensus.cc:348] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.298563 30317 raft_consensus.cc:729] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c301bdb7c55451e96793f78aec269ae, State: Initialized, Role: FOLLOWER
I20260923 18:02:54.299096 30317 consensus_queue.cc:260] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.300222 30317 ts_tablet_manager.cc:1428] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Time spent starting tablet: real 0.005s user 0.007s sys 0.001s
I20260923 18:02:54.301018 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.280979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:46959 (local address 127.15.139.190:44585)
0923 18:02:54.281480 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:54.281500 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:54.281974 (+ 474us) server_negotiation.cc:407] Connection header received
0923 18:02:54.283625 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:54.283642 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:54.284071 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:54.284475 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:54.286217 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.287839 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.290118 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.290895 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.291099 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.294620 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:54.294627 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:54.294645 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:54.294683 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:54.297054 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:54.297732 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:54.297738 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:54.297745 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:54.297909 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:54.298390 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:54.298394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:54.298396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:54.298885 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:54.299019 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:54.299763 (+ 744us) server_negotiation.cc:299] Negotiation successful
0923 18:02:54.300080 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":121,"threads_started":1}
I20260923 18:02:54.301077 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.280888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:46959)
0923 18:02:54.281760 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:54.281818 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:54.283071 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:54.284610 (+ 1539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:54.284622 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:54.285200 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:54.285956 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.285982 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.288039 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.288046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.289945 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.289962 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.291095 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.291102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.291363 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.292178 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:54.292200 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:54.294372 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:54.297207 (+ 2835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:54.297213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:54.297227 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:54.297594 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:54.298036 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:54.298039 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:54.298042 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:54.298230 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:54.299023 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:54.299030 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:54.299337 (+ 307us) client_negotiation.cc:769] Sending connection context
0923 18:02:54.299595 (+ 258us) client_negotiation.cc:240] Negotiation successful
0923 18:02:54.300001 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":129,"threads_started":1}
I20260923 18:02:54.303675 30425 heartbeater.cc:347] Connected to a master server at 127.15.139.190:44585
I20260923 18:02:54.303965 30425 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:54.304541 30425 heartbeater.cc:511] Master 127.15.139.190:44585 requested a full tablet report, sending...
I20260923 18:02:54.307323 30237 ts_manager.cc:194] Registered new tserver with Master: 6c301bdb7c55451e96793f78aec269ae (127.15.139.129:36979)
I20260923 18:02:54.310940 30237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:46959
I20260923 18:02:54.313628 30425 heartbeater.cc:503] Master 127.15.139.190:44585 was elected leader, sending a full tablet report...
Sep 23 18:02:54 dist-test-slave-vr57 krb5kdc[29656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186574, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.129@KRBTEST.COM
I20260923 18:02:54.322082 30433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.310382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38222 (local address 127.15.139.129:36979)
0923 18:02:54.311119 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:54.311132 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:54.311157 (+ 25us) server_negotiation.cc:407] Connection header received
0923 18:02:54.311229 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:54.311234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:54.311435 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:54.311618 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:54.312455 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.313484 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.314832 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.315333 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.315428 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.318208 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:54.318217 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:54.318227 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:54.318250 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:54.320000 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:54.320603 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:54.320609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:54.320611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:54.320712 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:54.321062 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:54.321065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:54.321066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:54.321301 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:54.321418 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:54.321802 (+ 384us) server_negotiation.cc:299] Negotiation successful
0923 18:02:54.321945 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":134,"threads_started":1}
I20260923 18:02:54.322134 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.310262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:36979 (local address 127.0.0.1:38222)
0923 18:02:54.310744 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:54.310798 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:54.310940 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:54.311641 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:54.311647 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:54.312158 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:54.312312 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.312321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.313624 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.313628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.314678 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.314691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.315446 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.315450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.315616 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.316217 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:54.316235 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:54.318055 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:54.320118 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:54.320128 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:54.320134 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:54.320454 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:54.320827 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:54.320830 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:54.320832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:54.320939 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:54.321421 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:54.321428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:54.321568 (+ 140us) client_negotiation.cc:769] Sending connection context
0923 18:02:54.321743 (+ 175us) client_negotiation.cc:240] Negotiation successful
0923 18:02:54.321953 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":129,"threads_started":1}
I20260923 18:02:54.323483 30359 tablet_service.cc:1514] Processing DeleteTablet for tablet 647415fa3d8d444cadedaa4a31e7e8e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:02:53 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:38222
I20260923 18:02:54.324657 30434 tablet_replica.cc:330] stopping tablet replica
I20260923 18:02:54.325038 30434 raft_consensus.cc:2229] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:02:54.325340 30434 raft_consensus.cc:2258] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:02:54.326599 30434 ts_tablet_manager.cc:1905] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:02:54.331698 30434 ts_tablet_manager.cc:1918] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:02:54.331830 30434 log.cc:1196] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/647415fa3d8d444cadedaa4a31e7e8e5
I20260923 18:02:54.332158 30434 ts_tablet_manager.cc:1939] T 647415fa3d8d444cadedaa4a31e7e8e5 P 6c301bdb7c55451e96793f78aec269ae: Deleting consensus metadata
I20260923 18:02:54.334456 30222 catalog_manager.cc:4841] TS 6c301bdb7c55451e96793f78aec269ae (127.15.139.129:36979): tablet 647415fa3d8d444cadedaa4a31e7e8e5 (table test-table [id=430a335964a14ee1b5a818884fa69006]) successfully deleted
W20260923 18:02:54.413740 30427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:54.414077 30427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:54.414125 30427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:54.418134 30427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:54.418200 30427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:54.418265 30427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:02:54.423683 30427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:40369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=43917
--tserver_master_addrs=127.15.139.190:44585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:54.424707 30427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:54.425724 30427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:54.435509 30442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:54.435547 30439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:54.435556 30440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:54.435909 30427 server_base.cc:1034] running on GCE node
I20260923 18:02:54.436342 30427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:54.436992 30427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:54.438196 30427 hybrid_clock.cc:648] HybridClock initialized: now 1790186574438172 us; error 40 us; skew 500 ppm
Sep 23 18:02:54 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186574, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:54.441076 30427 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:02:54.442056 30427 webserver.cc:460] Webserver started at http://127.15.139.130:43917/ using document root <none> and password file <none>
I20260923 18:02:54.442659 30427 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:54.442770 30427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:54.446628 30427 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:02:54.448945 30449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:54.450374 30427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:02:54.450505 30427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "a1f901ffff9e44949af805dd382c630f"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "d5caf35596f566cca7182e31792477b7"
server_key_iv: "e7b0041ce10e878c703fa59310a47887"
server_key_version: "encryptionkey@0"
I20260923 18:02:54.451062 30427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:54.467093 30427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:54.470018 30427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:54.470223 30427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:54.470801 30427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:54.471928 30456 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 18:02:54.476017 30427 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 18:02:54.476100 30427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260923 18:02:54.476183 30427 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 18:02:54.477797 30427 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 18:02:54.477874 30427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260923 18:02:54.478121 30456 tablet_bootstrap.cc:492] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Bootstrap starting.
I20260923 18:02:54.484122 30456 log.cc:824] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:54.488468 30427 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:40369
I20260923 18:02:54.488502 30564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:40369 every 8 connection(s)
I20260923 18:02:54.489349 30427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:02:54.492954 30456 tablet_bootstrap.cc:492] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 18:02:54.493391 30456 tablet_bootstrap.cc:492] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Bootstrap complete.
I20260923 18:02:54.493767 30456 ts_tablet_manager.cc:1397] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.004s
I20260923 18:02:54.496572 30456 raft_consensus.cc:348] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.497103 30456 raft_consensus.cc:729] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1f901ffff9e44949af805dd382c630f, State: Initialized, Role: FOLLOWER
I20260923 18:02:54.497148 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 30427
I20260923 18:02:54.497606 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:35579
--local_ip_for_outbound_sockets=127.15.139.131
--tserver_master_addrs=127.15.139.190:44585
--webserver_port=38927
--webserver_interface=127.15.139.131
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:02:54.497586 30456 consensus_queue.cc:260] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.498472 30456 ts_tablet_manager.cc:1428] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
Sep 23 18:02:54 dist-test-slave-vr57 krb5kdc[29656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186574, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:54.504619 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.491304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:45597 (local address 127.15.139.190:44585)
0923 18:02:54.491457 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:54.491462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:54.492108 (+ 646us) server_negotiation.cc:407] Connection header received
0923 18:02:54.493232 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:54.493235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:54.493286 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:54.493367 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:54.494775 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.495297 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.496941 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.497297 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.497422 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.501021 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:54.501025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:54.501028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:54.501059 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:54.502503 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:54.503008 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:54.503011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:54.503013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:54.503096 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:54.503489 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:54.503493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:54.503495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:54.503710 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:54.503812 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:54.504299 (+ 487us) server_negotiation.cc:299] Negotiation successful
0923 18:02:54.504451 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 18:02:54.505363 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.491387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:45597)
0923 18:02:54.491930 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:54.491981 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:54.493020 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:54.493525 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:54.493533 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:54.494017 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:54.494631 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.494649 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.495447 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.495451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.496779 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.496789 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.497558 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.497562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.497796 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.498925 (+ 1129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:54.498950 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:54.500842 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:54.502619 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:54.502625 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:54.502637 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:54.502899 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:54.503211 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:54.503214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:54.503215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:54.503383 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:54.503823 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:54.503831 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:54.504063 (+ 232us) client_negotiation.cc:769] Sending connection context
0923 18:02:54.504241 (+ 178us) client_negotiation.cc:240] Negotiation successful
0923 18:02:54.504560 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":128,"threads_started":1}
I20260923 18:02:54.506546 30565 heartbeater.cc:347] Connected to a master server at 127.15.139.190:44585
I20260923 18:02:54.506786 30565 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:54.507289 30565 heartbeater.cc:511] Master 127.15.139.190:44585 requested a full tablet report, sending...
I20260923 18:02:54.509074 30237 ts_manager.cc:194] Registered new tserver with Master: a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369)
I20260923 18:02:54.510339 30237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:45597
I20260923 18:02:54.512641 30565 heartbeater.cc:503] Master 127.15.139.190:44585 was elected leader, sending a full tablet report...
I20260923 18:02:54.516057 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.510028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:40369 (local address 127.0.0.1:53452)
0923 18:02:54.510184 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:54.510195 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:54.510257 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:54.511858 (+ 1601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:54.511861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:54.511883 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:54.511980 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.511988 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.513274 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.513278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.514613 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.514624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.515420 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.515423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.515697 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.515764 (+ 67us) client_negotiation.cc:769] Sending connection context
0923 18:02:54.515840 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 18:02:54.515935 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260923 18:02:54.516134 30570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.510663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.15.139.130:40369)
0923 18:02:54.511024 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:54.511035 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:54.511057 (+ 22us) server_negotiation.cc:407] Connection header received
0923 18:02:54.511128 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:54.511133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:54.511685 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:54.511859 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:54.512105 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.513149 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.514744 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.515275 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.515431 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.515605 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:54.515892 (+ 287us) server_negotiation.cc:299] Negotiation successful
0923 18:02:54.515995 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":138,"threads_started":1}
I20260923 18:02:54.517310 30498 tablet_service.cc:1514] Processing DeleteTablet for tablet 647415fa3d8d444cadedaa4a31e7e8e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:02:53 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:53452
I20260923 18:02:54.518249 30571 tablet_replica.cc:330] stopping tablet replica
I20260923 18:02:54.518559 30571 raft_consensus.cc:2229] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:02:54.518819 30571 raft_consensus.cc:2258] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:02:54.519878 30571 ts_tablet_manager.cc:1905] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:02:54.523487 30571 ts_tablet_manager.cc:1918] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:02:54.523638 30571 log.cc:1196] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/647415fa3d8d444cadedaa4a31e7e8e5
I20260923 18:02:54.523936 30571 ts_tablet_manager.cc:1939] T 647415fa3d8d444cadedaa4a31e7e8e5 P a1f901ffff9e44949af805dd382c630f: Deleting consensus metadata
I20260923 18:02:54.525472 30222 catalog_manager.cc:4841] TS a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369): tablet 647415fa3d8d444cadedaa4a31e7e8e5 (table test-table [id=430a335964a14ee1b5a818884fa69006]) successfully deleted
W20260923 18:02:54.619115 30569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:54.619438 30569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:54.619565 30569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:54.623778 30569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:54.623870 30569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:54.623957 30569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:02:54.629539 30569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:35579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=38927
--tserver_master_addrs=127.15.139.190:44585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:54.630650 30569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:54.631937 30569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:54.641767 30579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:54.641760 30577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:54.641791 30576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:54.641947 30569 server_base.cc:1034] running on GCE node
I20260923 18:02:54.642578 30569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:54.643263 30569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:54.644450 30569 hybrid_clock.cc:648] HybridClock initialized: now 1790186574644422 us; error 39 us; skew 500 ppm
Sep 23 18:02:54 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186574, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:54.647228 30569 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:02:54.648180 30569 webserver.cc:460] Webserver started at http://127.15.139.131:38927/ using document root <none> and password file <none>
I20260923 18:02:54.648810 30569 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:54.648923 30569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:54.652974 30569 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:02:54.655135 30586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:54.656523 30569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:02:54.656692 30569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "8975ed20d415485ba322c4d9be7ef71d"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "cc21f2117362e23cfecaabf8c063adfa"
server_key_iv: "f2aad0cd4bed0d2088a95f2ac94b53cb"
server_key_version: "encryptionkey@0"
I20260923 18:02:54.657263 30569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:54.672763 30569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:54.675836 30569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:54.676038 30569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:54.676656 30569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:54.677479 30569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:02:54.677551 30569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:54.677623 30569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:02:54.677665 30569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:54.686728 30569 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:35579
I20260923 18:02:54.686806 30699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:35579 every 8 connection(s)
I20260923 18:02:54.687805 30569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:02:54.695574 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 30569
Sep 23 18:02:54 dist-test-slave-vr57 krb5kdc[29656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186574, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:02:54.702369 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.689774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:39225 (local address 127.15.139.190:44585)
0923 18:02:54.689947 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:54.689950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:54.690738 (+ 788us) server_negotiation.cc:407] Connection header received
0923 18:02:54.692234 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:54.692238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:54.692298 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:54.692399 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:54.693558 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.694100 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.695295 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.695687 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.695800 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.698312 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:54.698315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:54.698317 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:54.698339 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:54.700092 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:54.700602 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:54.700607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:54.700608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:54.700697 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:54.701045 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:54.701050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:54.701051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:54.701229 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:54.701345 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:54.701959 (+ 614us) server_negotiation.cc:299] Negotiation successful
0923 18:02:54.702140 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 18:02:54.703119 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.690156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:39225)
0923 18:02:54.690590 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:54.690632 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:54.692013 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:54.692584 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:54.692591 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:54.692951 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:54.693415 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.693427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.694229 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.694232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.695158 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.695168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.695883 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.695886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.696038 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.696686 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:54.696702 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:54.698135 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:54.700211 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:54.700216 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:54.700231 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:54.700496 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:54.700798 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:54.700802 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:54.700804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:54.700949 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:54.701353 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:54.701361 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:54.701608 (+ 247us) client_negotiation.cc:769] Sending connection context
0923 18:02:54.701874 (+ 266us) client_negotiation.cc:240] Negotiation successful
0923 18:02:54.702159 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":102,"threads_started":1}
I20260923 18:02:54.704391 30700 heartbeater.cc:347] Connected to a master server at 127.15.139.190:44585
I20260923 18:02:54.704656 30700 heartbeater.cc:464] Registering TS with master...
I20260923 18:02:54.704975 30703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.696305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51732 (local address 127.15.139.190:44585)
0923 18:02:54.696741 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:54.696746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:54.696856 (+ 110us) server_negotiation.cc:407] Connection header received
0923 18:02:54.697014 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:54.697017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:54.697068 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:54.697163 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:54.697940 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.698560 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.699713 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.699971 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.700082 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.701399 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:54.701403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:54.701405 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:54.701434 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:54.703165 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:54.703692 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:54.703695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:54.703697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:54.703784 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:54.704107 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:54.704110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:54.704112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:54.704321 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:54.704549 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:54.704694 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 18:02:54.704841 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":68,"threads_started":1}
I20260923 18:02:54.705206 30700 heartbeater.cc:511] Master 127.15.139.190:44585 requested a full tablet report, sending...
I20260923 18:02:54.706378 30237 ts_manager.cc:194] Registered new tserver with Master: 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579)
I20260923 18:02:54.707027 30237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:39225
I20260923 18:02:54.707274 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:02:54.711678 30703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.707903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51748 (local address 127.15.139.190:44585)
0923 18:02:54.708014 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:54.708018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:54.708307 (+ 289us) server_negotiation.cc:407] Connection header received
0923 18:02:54.708377 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:54.708379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:54.708461 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:54.708541 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:02:54.708826 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.709326 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.710167 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.710387 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.710485 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.710885 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:02:54.711244 (+ 359us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:02:54.711334 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:54.711459 (+ 125us) server_negotiation.cc:299] Negotiation successful
0923 18:02:54.711572 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 18:02:54.712389 30237 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:02:54.715673 30237 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:02:54.724026 30498 tablet_service.cc:1467] Processing CreateTablet for tablet 0fd8cd9512af4d55bcdb52da5ce532b7 (DEFAULT_TABLE table=test-table [id=5ec2759bebd748bfafed16c1583cb77b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:02:54.724004 30359 tablet_service.cc:1467] Processing CreateTablet for tablet 0fd8cd9512af4d55bcdb52da5ce532b7 (DEFAULT_TABLE table=test-table [id=5ec2759bebd748bfafed16c1583cb77b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:02:54.724680 30498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fd8cd9512af4d55bcdb52da5ce532b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:54.724700 30359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fd8cd9512af4d55bcdb52da5ce532b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:02:54.727423 30456 tablet_bootstrap.cc:492] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f: Bootstrap starting.
I20260923 18:02:54.727594 30317 tablet_bootstrap.cc:492] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae: Bootstrap starting.
I20260923 18:02:54.728513 30456 tablet_bootstrap.cc:654] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:54.728580 30317 tablet_bootstrap.cc:654] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:54.729553 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.722987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:35579 (local address 127.0.0.1:58786)
0923 18:02:54.723134 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:54.723152 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:54.723248 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:54.724217 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:54.724220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:54.724239 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:54.724340 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.724359 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.726228 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.726231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.727437 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.727451 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.729061 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.729066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.729163 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.729213 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 18:02:54.729279 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 18:02:54.729369 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260923 18:02:54.730137 30456 tablet_bootstrap.cc:492] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f: No bootstrap required, opened a new log
I20260923 18:02:54.730232 30456 ts_tablet_manager.cc:1397] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:02:54.730254 30317 tablet_bootstrap.cc:492] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae: No bootstrap required, opened a new log
I20260923 18:02:54.730342 30317 ts_tablet_manager.cc:1397] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:02:54.730361 30709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.723183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58786 (local address 127.15.139.131:35579)
0923 18:02:54.723656 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:54.723671 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:54.723700 (+ 29us) server_negotiation.cc:407] Connection header received
0923 18:02:54.723787 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:54.723794 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:54.724036 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:54.724246 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:54.724495 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.726058 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.727673 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.728863 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.729890 (+ 1027us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.730044 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:54.730101 (+ 57us) server_negotiation.cc:299] Negotiation successful
0923 18:02:54.730206 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":211,"threads_started":1}
I20260923 18:02:54.730732 30456 raft_consensus.cc:348] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.730857 30456 raft_consensus.cc:374] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:54.730895 30456 raft_consensus.cc:729] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1f901ffff9e44949af805dd382c630f, State: Initialized, Role: FOLLOWER
I20260923 18:02:54.730902 30317 raft_consensus.cc:348] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.731026 30317 raft_consensus.cc:374] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:54.731014 30456 consensus_queue.cc:260] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.731060 30317 raft_consensus.cc:729] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c301bdb7c55451e96793f78aec269ae, State: Initialized, Role: FOLLOWER
I20260923 18:02:54.731263 30456 ts_tablet_manager.cc:1428] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:02:54.731231 30317 consensus_queue.cc:260] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.732426 30317 ts_tablet_manager.cc:1428] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260923 18:02:54.734345 30634 tablet_service.cc:1467] Processing CreateTablet for tablet 0fd8cd9512af4d55bcdb52da5ce532b7 (DEFAULT_TABLE table=test-table [id=5ec2759bebd748bfafed16c1583cb77b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:02:54.735460 30634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fd8cd9512af4d55bcdb52da5ce532b7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 18:02:54.740447 30566 tablet.cc:2369] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:02:54.742753 30712 tablet_bootstrap.cc:492] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d: Bootstrap starting.
I20260923 18:02:54.745163 30712 tablet_bootstrap.cc:654] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d: Neither blocks nor log segments found. Creating new log.
I20260923 18:02:54.745929 30712 log.cc:824] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:54.747953 30712 tablet_bootstrap.cc:492] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d: No bootstrap required, opened a new log
I20260923 18:02:54.748113 30712 ts_tablet_manager.cc:1397] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260923 18:02:54.750977 30712 raft_consensus.cc:348] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.751192 30712 raft_consensus.cc:374] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:02:54.751231 30712 raft_consensus.cc:729] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8975ed20d415485ba322c4d9be7ef71d, State: Initialized, Role: FOLLOWER
I20260923 18:02:54.751663 30712 consensus_queue.cc:260] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.752310 30700 heartbeater.cc:503] Master 127.15.139.190:44585 was elected leader, sending a full tablet report...
I20260923 18:02:54.752614 30712 ts_tablet_manager.cc:1428] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260923 18:02:54.779099 30426 tablet.cc:2369] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:02:54.832697 30714 raft_consensus.cc:482] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:02:54.832980 30714 raft_consensus.cc:504] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.834306 30714 leader_election.cc:288] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c301bdb7c55451e96793f78aec269ae (127.15.139.129:36979), a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369)
I20260923 18:02:54.839742 30570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.835044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:48943 (local address 127.15.139.130:40369)
0923 18:02:54.835191 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:54.835195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:54.835350 (+ 155us) server_negotiation.cc:407] Connection header received
0923 18:02:54.835497 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:54.835532 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:54.835592 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:54.835656 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:54.835938 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.836668 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.838321 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.838921 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.839061 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.839122 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:54.839486 (+ 364us) server_negotiation.cc:299] Negotiation successful
0923 18:02:54.839601 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 18:02:54.839785 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.834631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:36979 (local address 127.15.139.131:45465)
0923 18:02:54.834801 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:54.834820 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:54.834939 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:54.835426 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:54.835431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:54.835450 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:54.835616 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.835623 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.836739 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.836743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.838188 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.838196 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.839142 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.839145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.839269 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.839331 (+ 62us) client_negotiation.cc:769] Sending connection context
0923 18:02:54.839433 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 18:02:54.839582 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":140}
I20260923 18:02:54.839797 30716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.834927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:40369 (local address 127.15.139.131:48943)
0923 18:02:54.835258 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:54.835270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:54.835386 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:54.835684 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:54.835689 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:54.835712 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:02:54.835819 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.835823 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.836763 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.836766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.838185 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:54.838196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.839107 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.839114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:54.839334 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.839358 (+ 24us) client_negotiation.cc:769] Sending connection context
0923 18:02:54.839436 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 18:02:54.839577 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":89,"thread_start_us":121,"threads_started":1}
I20260923 18:02:54.840035 30715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.834774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:45465 (local address 127.15.139.129:36979)
0923 18:02:54.835070 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:54.835095 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:54.835113 (+ 18us) server_negotiation.cc:407] Connection header received
0923 18:02:54.835172 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:54.835175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:54.835251 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:54.835407 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:02:54.835791 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.836600 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.838321 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.839015 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.839124 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.839277 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:54.839487 (+ 210us) server_negotiation.cc:299] Negotiation successful
0923 18:02:54.839868 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":104,"threads_started":1}
I20260923 18:02:54.840549 30518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd8cd9512af4d55bcdb52da5ce532b7" candidate_uuid: "8975ed20d415485ba322c4d9be7ef71d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1f901ffff9e44949af805dd382c630f" is_pre_election: true
I20260923 18:02:54.840543 30379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd8cd9512af4d55bcdb52da5ce532b7" candidate_uuid: "8975ed20d415485ba322c4d9be7ef71d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c301bdb7c55451e96793f78aec269ae" is_pre_election: true
I20260923 18:02:54.840853 30518 raft_consensus.cc:2454] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8975ed20d415485ba322c4d9be7ef71d in term 0.
I20260923 18:02:54.840874 30379 raft_consensus.cc:2454] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8975ed20d415485ba322c4d9be7ef71d in term 0.
I20260923 18:02:54.841347 30588 leader_election.cc:302] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8975ed20d415485ba322c4d9be7ef71d, a1f901ffff9e44949af805dd382c630f; no voters:
I20260923 18:02:54.841636 30714 raft_consensus.cc:2781] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:02:54.841712 30714 raft_consensus.cc:482] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:02:54.841742 30714 raft_consensus.cc:3037] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:54.842869 30714 raft_consensus.cc:504] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.843207 30714 leader_election.cc:288] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [CANDIDATE]: Term 1 election: Requested vote from peers 6c301bdb7c55451e96793f78aec269ae (127.15.139.129:36979), a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369)
I20260923 18:02:54.843748 30379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd8cd9512af4d55bcdb52da5ce532b7" candidate_uuid: "8975ed20d415485ba322c4d9be7ef71d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c301bdb7c55451e96793f78aec269ae"
I20260923 18:02:54.843755 30518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd8cd9512af4d55bcdb52da5ce532b7" candidate_uuid: "8975ed20d415485ba322c4d9be7ef71d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1f901ffff9e44949af805dd382c630f"
I20260923 18:02:54.843892 30379 raft_consensus.cc:3037] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:54.843909 30518 raft_consensus.cc:3037] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:02:54.845260 30379 raft_consensus.cc:2454] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8975ed20d415485ba322c4d9be7ef71d in term 1.
I20260923 18:02:54.845289 30518 raft_consensus.cc:2454] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8975ed20d415485ba322c4d9be7ef71d in term 1.
I20260923 18:02:54.845535 30588 leader_election.cc:302] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c301bdb7c55451e96793f78aec269ae, 8975ed20d415485ba322c4d9be7ef71d; no voters:
I20260923 18:02:54.845768 30714 raft_consensus.cc:2781] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:02:54.846032 30714 raft_consensus.cc:686] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 1 LEADER]: Becoming Leader. State: Replica: 8975ed20d415485ba322c4d9be7ef71d, State: Running, Role: LEADER
I20260923 18:02:54.846421 30714 consensus_queue.cc:237] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:54.849807 30236 catalog_manager.cc:5495] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131). New cstate: current_term: 1 leader_uuid: "8975ed20d415485ba322c4d9be7ef71d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } health_report { overall_health: UNKNOWN } } }
I20260923 18:02:54.883687 30709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.879929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58802 (local address 127.15.139.131:35579)
0923 18:02:54.880068 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:54.880071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:54.880085 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:02:54.880285 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:54.880288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:54.880354 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:54.880458 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:02:54.880673 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.881250 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.882071 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:54.882355 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:54.882451 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:54.882753 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:02:54.883231 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:02:54.883341 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:54.883391 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 18:02:54.883445 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260923 18:02:54.889858 30518 raft_consensus.cc:1261] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f [term 1 FOLLOWER]: Refusing update from remote peer 8975ed20d415485ba322c4d9be7ef71d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:02:54.889878 30379 raft_consensus.cc:1261] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Refusing update from remote peer 8975ed20d415485ba322c4d9be7ef71d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:02:54.890523 30714 consensus_queue.cc:1035] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:02:54.890767 30717 consensus_queue.cc:1035] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:02:54.896591 30720 mvcc.cc:205] Tried to move back new op lower bound from 7332604210741317632 to 7332604210573570048. Current Snapshot: MvccSnapshot[applied={T|T < 7332604210741317632}]
I20260923 18:02:54.896924 30721 mvcc.cc:205] Tried to move back new op lower bound from 7332604210741317632 to 7332604210573570048. Current Snapshot: MvccSnapshot[applied={T|T < 7332604210741317632}]
I20260923 18:02:54.904505 30236 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:02:54.904678 30236 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:02:54.907085 30236 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Sending DeleteTablet for 3 replicas of tablet 0fd8cd9512af4d55bcdb52da5ce532b7
I20260923 18:02:54.907881 30359 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fd8cd9512af4d55bcdb52da5ce532b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:02:54 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:38222
I20260923 18:02:54.907846 30634 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fd8cd9512af4d55bcdb52da5ce532b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:02:54 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:58786
I20260923 18:02:54.908180 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 30275
I20260923 18:02:54.908828 30731 tablet_replica.cc:330] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d: stopping tablet replica
I20260923 18:02:54.908913 30498 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fd8cd9512af4d55bcdb52da5ce532b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:02:54 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:53452
I20260923 18:02:54.909116 30571 tablet_replica.cc:330] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f: stopping tablet replica
I20260923 18:02:54.909168 30731 raft_consensus.cc:2229] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:02:54.909230 30571 raft_consensus.cc:2229] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:02:54.909363 30571 raft_consensus.cc:2258] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:02:54.909535 30731 raft_consensus.cc:2258] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:02:54.910488 30571 ts_tablet_manager.cc:1905] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:02:54.910912 30731 ts_tablet_manager.cc:1905] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:02:54.913146 30571 ts_tablet_manager.cc:1918] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:02:54.913234 30571 log.cc:1196] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/0fd8cd9512af4d55bcdb52da5ce532b7
I20260923 18:02:54.913523 30571 ts_tablet_manager.cc:1939] T 0fd8cd9512af4d55bcdb52da5ce532b7 P a1f901ffff9e44949af805dd382c630f: Deleting consensus metadata
I20260923 18:02:54.913695 30731 ts_tablet_manager.cc:1918] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:02:54.913803 30731 log.cc:1196] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/0fd8cd9512af4d55bcdb52da5ce532b7
I20260923 18:02:54.914155 30731 ts_tablet_manager.cc:1939] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 8975ed20d415485ba322c4d9be7ef71d: Deleting consensus metadata
I20260923 18:02:54.914371 30222 catalog_manager.cc:4841] TS a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369): tablet 0fd8cd9512af4d55bcdb52da5ce532b7 (table test-table [id=5ec2759bebd748bfafed16c1583cb77b]) successfully deleted
I20260923 18:02:54.915251 30222 catalog_manager.cc:4841] TS 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579): tablet 0fd8cd9512af4d55bcdb52da5ce532b7 (table test-table [id=5ec2759bebd748bfafed16c1583cb77b]) successfully deleted
W20260923 18:02:54.916254 30222 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:36979 (error 108)
I20260923 18:02:54.916504 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 30427
I20260923 18:02:54.918105 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:54.917749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:36979 (local address 127.0.0.1:38224)
0923 18:02:54.917905 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:54.918000 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:36979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 18:02:54.918289 30222 catalog_manager.cc:4584] TS 6c301bdb7c55451e96793f78aec269ae (127.15.139.129:36979): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0fd8cd9512af4d55bcdb52da5ce532b7: Network error: Client connection negotiation failed: client connection to 127.15.139.129:36979: connect: Connection refused (error 111)
I20260923 18:02:54.924122 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 30569
I20260923 18:02:54.930680 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 30204
I20260923 18:02:54.937366 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:44585
--webserver_interface=127.15.139.190
--webserver_port=36605
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:44585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:55.062089 30733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:55.062372 30733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:55.062413 30733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:55.066511 30733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:02:55.066576 30733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:55.066596 30733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:55.066613 30733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:02:55.066628 30733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:02:55.072108 30733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:44585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:44585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=36605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:55.073151 30733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:55.074213 30733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:55.082880 30741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:55.082888 30738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:55.082989 30739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:55.083086 30733 server_base.cc:1034] running on GCE node
I20260923 18:02:55.083858 30733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:55.084961 30733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:55.086149 30733 hybrid_clock.cc:648] HybridClock initialized: now 1790186575086137 us; error 40 us; skew 500 ppm
Sep 23 18:02:55 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:55.089905 30733 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:02:55.090873 30733 webserver.cc:460] Webserver started at http://127.15.139.190:36605/ using document root <none> and password file <none>
I20260923 18:02:55.091439 30733 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:55.091548 30733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:55.095347 30733 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:02:55.097759 30748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:55.099074 30733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:02:55.099244 30733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "2075dc7b9ae049d582addbf085ac51e9"
format_stamp: "Formatted at 2026-09-23 18:02:52 on dist-test-slave-vr57"
server_key: "4cfabfcba9a2a5703b27371f63a15bfc"
server_key_iv: "cafcacf1f54cd0dafb2df8de37c978a2"
server_key_version: "encryptionkey@0"
I20260923 18:02:55.099932 30733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:55.118083 30733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:55.121829 30733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:55.122040 30733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:55.129444 30733 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:44585
I20260923 18:02:55.129460 30800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:44585 every 8 connection(s)
I20260923 18:02:55.130595 30733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:02:55.133989 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 30733
I20260923 18:02:55.134502 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:36979
--local_ip_for_outbound_sockets=127.15.139.129
--tserver_master_addrs=127.15.139.190:44585
--webserver_port=40997
--webserver_interface=127.15.139.129
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:02:55.135303 30801 sys_catalog.cc:263] Verifying existing consensus state
I20260923 18:02:55.137333 30801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Bootstrap starting.
I20260923 18:02:55.155323 30801 log.cc:824] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:55.166517 30801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 18:02:55.166970 30801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9: Bootstrap complete.
I20260923 18:02:55.171309 30801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } }
I20260923 18:02:55.171788 30801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2075dc7b9ae049d582addbf085ac51e9, State: Initialized, Role: FOLLOWER
I20260923 18:02:55.172348 30801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } }
I20260923 18:02:55.172464 30801 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:02:55.172549 30801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:02:55.172664 30801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 2 FOLLOWER]: Advancing to term 3
I20260923 18:02:55.174566 30801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } }
I20260923 18:02:55.174901 30801 leader_election.cc:302] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2075dc7b9ae049d582addbf085ac51e9; no voters:
I20260923 18:02:55.175246 30801 leader_election.cc:288] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 18:02:55.175383 30805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 18:02:55.175645 30805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [term 3 LEADER]: Becoming Leader. State: Replica: 2075dc7b9ae049d582addbf085ac51e9, State: Running, Role: LEADER
I20260923 18:02:55.175972 30805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } }
I20260923 18:02:55.176509 30801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:02:55.177520 30806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2075dc7b9ae049d582addbf085ac51e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } } }
I20260923 18:02:55.177582 30807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2075dc7b9ae049d582addbf085ac51e9. Latest consensus state: current_term: 3 leader_uuid: "2075dc7b9ae049d582addbf085ac51e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2075dc7b9ae049d582addbf085ac51e9" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 44585 } } }
I20260923 18:02:55.177656 30806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:55.177696 30807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2075dc7b9ae049d582addbf085ac51e9 [sys.catalog]: This master's current role is: LEADER
I20260923 18:02:55.178021 30814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:02:55.183212 30814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:02:55.183858 30814 catalog_manager.cc:1259] Loaded cluster ID: d341bc3404124bcdbdedf713972d3d66
I20260923 18:02:55.183918 30814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260923 18:02:55.184839 30814 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fda7c4d39ad google::LogMessage::Fail() at ??:0
@ 0x7fda7c4d688a google::LogMessage::SendToLog() at ??:0
@ 0x7fda7c4d34de google::LogMessage::Flush() at ??:0
@ 0x7fda7c4d4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fda86e16fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fda86e17d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fda86e14579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fda86e58640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fda7e7253d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fda7e725c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fda7e7273f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fda7e716512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fda7fd5a6db start_thread at ??:0
@ 0x7fda7afd161f clone at ??:0
W20260923 18:02:55.270907 30803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:55.271250 30803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:55.271381 30803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:55.276163 30803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:55.276260 30803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:55.276397 30803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:02:55.282528 30803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:36979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=40997
--tserver_master_addrs=127.15.139.190:44585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:55.283859 30803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:55.284991 30803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:55.295856 30842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:55.295878 30843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:55.295855 30845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:55.296607 30803 server_base.cc:1034] running on GCE node
I20260923 18:02:55.297092 30803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:55.297837 30803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:55.299033 30803 hybrid_clock.cc:648] HybridClock initialized: now 1790186575299034 us; error 48 us; skew 500 ppm
Sep 23 18:02:55 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:55.302163 30803 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:02:55.303140 30803 webserver.cc:460] Webserver started at http://127.15.139.129:40997/ using document root <none> and password file <none>
I20260923 18:02:55.303822 30803 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:55.303891 30803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:55.308377 30803 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260923 18:02:55.310961 30852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:55.312422 30803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:02:55.312579 30803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "6c301bdb7c55451e96793f78aec269ae"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "92b2c59d391aa99f3e322de2aa4e5123"
server_key_iv: "97841cda20eb4b8056f98687409dc7f3"
server_key_version: "encryptionkey@0"
I20260923 18:02:55.313143 30803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:55.330484 30803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:55.333915 30803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:55.334128 30803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:55.334748 30803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:55.335952 30859 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 18:02:55.341943 30803 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 18:02:55.342036 30803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260923 18:02:55.342123 30803 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 18:02:55.343900 30803 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 18:02:55.343972 30803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260923 18:02:55.344314 30859 tablet_bootstrap.cc:492] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae: Bootstrap starting.
I20260923 18:02:55.354768 30803 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:36979
I20260923 18:02:55.354787 30966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:36979 every 8 connection(s)
I20260923 18:02:55.355834 30803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:02:55.362087 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 30803
I20260923 18:02:55.362533 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:40369
--local_ip_for_outbound_sockets=127.15.139.130
--tserver_master_addrs=127.15.139.190:44585
--webserver_port=43917
--webserver_interface=127.15.139.130
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:02:55.370685 30859 log.cc:824] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae: Log is configured to *not* fsync() on all Append() calls
I20260923 18:02:55.378110 30859 tablet_bootstrap.cc:492] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 18:02:55.378515 30859 tablet_bootstrap.cc:492] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae: Bootstrap complete.
I20260923 18:02:55.379756 30859 ts_tablet_manager.cc:1397] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae: Time spent bootstrapping tablet: real 0.036s user 0.020s sys 0.013s
I20260923 18:02:55.382248 30859 raft_consensus.cc:348] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:55.382689 30859 raft_consensus.cc:729] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c301bdb7c55451e96793f78aec269ae, State: Initialized, Role: FOLLOWER
I20260923 18:02:55.383189 30859 consensus_queue.cc:260] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:55.384434 30859 ts_tablet_manager.cc:1428] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
W20260923 18:02:55.466953 30969 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:02:55.357884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:35569)
0923 18:02:55.358378 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:55.358434 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:55.359284 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:55.466145 (+106861us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":146,"threads_started":1}
W20260923 18:02:55.467629 30967 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:44585 (0 consecutive failures): Network error: Failed to ping master at 127.15.139.190:44585: Client connection negotiation failed: client connection to 127.15.139.190:44585: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 18:02:55.468423 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:55.468128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:40437)
0923 18:02:55.468256 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:55.468348 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260923 18:02:55.469162 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:55.468951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:58017)
0923 18:02:55.469055 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:55.469111 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260923 18:02:55.469393 30967 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 18:02:55.515967 30970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:55.516288 30970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:55.516379 30970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:55.520963 30970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:55.521056 30970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:55.521153 30970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:02:55.526973 30970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:40369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=43917
--tserver_master_addrs=127.15.139.190:44585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:55.528139 30970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:55.529249 30970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:55.539582 30980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:55.539605 30977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:55.539628 30978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:55.539953 30970 server_base.cc:1034] running on GCE node
I20260923 18:02:55.540465 30970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:55.541254 30970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:55.542475 30970 hybrid_clock.cc:648] HybridClock initialized: now 1790186575542436 us; error 51 us; skew 500 ppm
Sep 23 18:02:55 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:55.545500 30970 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:02:55.546430 30970 webserver.cc:460] Webserver started at http://127.15.139.130:43917/ using document root <none> and password file <none>
I20260923 18:02:55.547019 30970 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:55.547086 30970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:55.551357 30970 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:02:55.553613 30987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:55.555042 30970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:02:55.555145 30970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "a1f901ffff9e44949af805dd382c630f"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "d5caf35596f566cca7182e31792477b7"
server_key_iv: "e7b0041ce10e878c703fa59310a47887"
server_key_version: "encryptionkey@0"
I20260923 18:02:55.555773 30970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:55.577838 30970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:55.581524 30970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:55.581697 30970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:55.582285 30970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:55.583214 30970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:02:55.583266 30970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:55.583333 30970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:02:55.583379 30970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:55.593286 30970 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:40369
I20260923 18:02:55.593405 31100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:40369 every 8 connection(s)
I20260923 18:02:55.594383 30970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:02:55.598358 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:55.596587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:56647)
0923 18:02:55.597051 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:55.597583 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":107,"threads_started":1}
W20260923 18:02:55.598865 31101 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:44585 (0 consecutive failures): Network error: Failed to ping master at 127.15.139.190:44585: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
I20260923 18:02:55.599484 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:55.599267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:49135)
0923 18:02:55.599374 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:55.599426 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 18:02:55.600292 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:55.600034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:58755)
0923 18:02:55.600166 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:55.600217 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260923 18:02:55.600579 31101 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 18:02:55.603650 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 30970
I20260923 18:02:55.604051 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:35579
--local_ip_for_outbound_sockets=127.15.139.131
--tserver_master_addrs=127.15.139.190:44585
--webserver_port=38927
--webserver_interface=127.15.139.131
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:02:55.726711 31104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:02:55.727038 31104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:02:55.727116 31104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:02:55.731256 31104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:02:55.731339 31104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:02:55.731434 31104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:02:55.736994 31104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:35579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=38927
--tserver_master_addrs=127.15.139.190:44585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:02:55.738065 31104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:02:55.739115 31104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:02:55.748850 31109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:55.748909 31104 server_base.cc:1034] running on GCE node
W20260923 18:02:55.748968 31110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:02:55.748869 31112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:02:55.749500 31104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:02:55.750201 31104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:02:55.751387 31104 hybrid_clock.cc:648] HybridClock initialized: now 1790186575751374 us; error 40 us; skew 500 ppm
Sep 23 18:02:55 dist-test-slave-vr57 krb5kdc[29656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:02:55.754143 31104 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:02:55.755167 31104 webserver.cc:460] Webserver started at http://127.15.139.131:38927/ using document root <none> and password file <none>
I20260923 18:02:55.755834 31104 fs_manager.cc:359] Metadata directory not provided
I20260923 18:02:55.755944 31104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:02:55.760113 31104 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:02:55.762326 31119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:55.763909 31104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:02:55.764068 31104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "8975ed20d415485ba322c4d9be7ef71d"
format_stamp: "Formatted at 2026-09-23 18:02:53 on dist-test-slave-vr57"
server_key: "cc21f2117362e23cfecaabf8c063adfa"
server_key_iv: "f2aad0cd4bed0d2088a95f2ac94b53cb"
server_key_version: "encryptionkey@0"
I20260923 18:02:55.764681 31104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:02:55.794028 31104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:02:55.797605 31104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:02:55.797878 31104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:02:55.798578 31104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:02:55.799491 31104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:02:55.799597 31104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:55.799677 31104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:02:55.799726 31104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:02:55.809649 31104 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:35579
I20260923 18:02:55.809670 31232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:35579 every 8 connection(s)
I20260923 18:02:55.810763 31104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:02:55.811559 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 31104
I20260923 18:02:55.814683 31236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:55.813116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:48505)
0923 18:02:55.813586 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:55.813987 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":132,"threads_started":1}
W20260923 18:02:55.815197 31233 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:44585 (0 consecutive failures): Network error: Failed to ping master at 127.15.139.190:44585: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
I20260923 18:02:55.815800 31236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:55.815597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:45365)
0923 18:02:55.815703 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:55.815747 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 18:02:55.816555 31236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:55.816312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:49181)
0923 18:02:55.816430 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:55.816486 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260923 18:02:55.816736 31233 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 18:02:56.470481 31237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:56.470039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:42047)
0923 18:02:56.470307 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:56.470368 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":85,"threads_started":1}
I20260923 18:02:56.601665 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:56.601199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:51783)
0923 18:02:56.601488 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:56.601557 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":80,"threads_started":1}
I20260923 18:02:56.817714 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:56.817337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:42121)
0923 18:02:56.817562 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:56.817616 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":67,"threads_started":1}
I20260923 18:02:57.139312 31240 raft_consensus.cc:482] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:02:57.139603 31240 raft_consensus.cc:504] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:57.140795 31240 leader_election.cc:288] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579), a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369)
Sep 23 18:02:57 dist-test-slave-vr57 krb5kdc[29656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.130@KRBTEST.COM
Sep 23 18:02:57 dist-test-slave-vr57 krb5kdc[29656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.131@KRBTEST.COM
I20260923 18:02:57.154960 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:57.141615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:58303 (local address 127.15.139.130:40369)
0923 18:02:57.141920 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:57.141931 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:57.141966 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:02:57.142614 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:57.142620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:57.143049 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:57.143372 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:57.144787 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:57.145760 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:57.147018 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:57.147699 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:57.147886 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:57.150258 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:57.150263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:57.150275 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:57.150301 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:57.152246 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:57.152878 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:57.152884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:57.152890 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:57.153006 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:57.153368 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:57.153373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:57.153378 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:57.153861 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:57.154002 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:57.154594 (+ 592us) server_negotiation.cc:299] Negotiation successful
0923 18:02:57.154819 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":95,"threads_started":1}
I20260923 18:02:57.155043 31241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:57.141065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:35579 (local address 127.15.139.129:39063)
0923 18:02:57.141346 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:57.141368 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:57.141499 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:57.143474 (+ 1975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:57.143481 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:57.144105 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:57.144600 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:57.144616 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:57.145926 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:57.145928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:57.146873 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:57.146882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:57.147668 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:57.147672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:57.147867 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:57.148638 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:57.148650 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:57.150440 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:57.152495 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:57.152498 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:57.152499 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:57.152764 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:57.153126 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:57.153129 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:57.153131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:57.153275 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:57.153937 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:57.153947 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:57.154247 (+ 300us) client_negotiation.cc:769] Sending connection context
0923 18:02:57.154468 (+ 221us) client_negotiation.cc:240] Negotiation successful
0923 18:02:57.154905 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":278,"spinlock_wait_cycles":11392,"thread_start_us":84,"threads_started":1}
I20260923 18:02:57.155043 31243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:57.141437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:40369 (local address 127.15.139.129:58303)
0923 18:02:57.141751 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:57.141772 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 18:02:57.141873 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:02:57.143474 (+ 1601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:02:57.143485 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:02:57.144117 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:02:57.144601 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:57.144616 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:57.145908 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:57.145913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:57.146870 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:02:57.146880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:02:57.147829 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:02:57.147833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:02:57.147937 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:57.148476 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:02:57.148492 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:02:57.150017 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:02:57.152367 (+ 2350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:57.152373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:57.152388 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:57.152757 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:57.153101 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:02:57.153108 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:02:57.153113 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:02:57.153263 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:02:57.154045 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:02:57.154048 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:02:57.154248 (+ 200us) client_negotiation.cc:769] Sending connection context
0923 18:02:57.154468 (+ 220us) client_negotiation.cc:240] Negotiation successful
0923 18:02:57.154844 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":129,"spinlock_wait_cycles":50176,"thread_start_us":70,"threads_started":1}
I20260923 18:02:57.155745 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:57.141216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:39063 (local address 127.15.139.131:35579)
0923 18:02:57.141777 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0923 18:02:57.141850 (+ 73us) server_negotiation.cc:399] Waiting for connection header
0923 18:02:57.141900 (+ 50us) server_negotiation.cc:407] Connection header received
0923 18:02:57.142615 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:02:57.142621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:02:57.143051 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:02:57.143828 (+ 777us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:02:57.144773 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:57.145760 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:57.147014 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:02:57.147543 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:02:57.147680 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:02:57.150620 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:02:57.150624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:02:57.150634 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:02:57.150659 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:02:57.152380 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:57.152880 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:57.152890 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:57.152894 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:57.153004 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:02:57.153401 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:02:57.153404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:02:57.153405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:02:57.153808 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:02:57.153946 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0923 18:02:57.155312 (+ 1366us) server_negotiation.cc:299] Negotiation successful
0923 18:02:57.155611 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":170,"threads_started":1}
W20260923 18:02:57.157823 30854 leader_election.cc:341] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579): Not found: Tablet not found: 0fd8cd9512af4d55bcdb52da5ce532b7
W20260923 18:02:57.158039 30854 leader_election.cc:341] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369): Not found: Tablet not found: 0fd8cd9512af4d55bcdb52da5ce532b7
I20260923 18:02:57.158107 30854 leader_election.cc:302] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c301bdb7c55451e96793f78aec269ae; no voters: 8975ed20d415485ba322c4d9be7ef71d, a1f901ffff9e44949af805dd382c630f
I20260923 18:02:57.158430 31240 raft_consensus.cc:2726] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 18:02:57.471676 31243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:57.471312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:33255)
0923 18:02:57.471459 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:57.471586 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260923 18:02:57.602960 31245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:57.602528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:42983)
0923 18:02:57.602792 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:57.602848 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":77,"threads_started":1}
I20260923 18:02:57.818929 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:57.818527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:39259)
0923 18:02:57.818773 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:57.818832 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260923 18:02:58.473016 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:58.472495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:36633)
0923 18:02:58.472828 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:58.472897 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":78,"threads_started":1}
I20260923 18:02:58.604808 31248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:58.604181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:58447)
0923 18:02:58.604561 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:58.604640 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":121,"threads_started":1}
I20260923 18:02:58.820407 31249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:58.819965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:57023)
0923 18:02:58.820236 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:58.820300 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":83,"threads_started":1}
I20260923 18:02:59.394951 31250 raft_consensus.cc:482] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:02:59.395082 31250 raft_consensus.cc:504] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:02:59.395639 31250 leader_election.cc:288] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579), a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369)
W20260923 18:02:59.396516 30854 leader_election.cc:341] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369): Not found: Tablet not found: 0fd8cd9512af4d55bcdb52da5ce532b7
W20260923 18:02:59.396624 30854 leader_election.cc:341] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579): Not found: Tablet not found: 0fd8cd9512af4d55bcdb52da5ce532b7
I20260923 18:02:59.396693 30854 leader_election.cc:302] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c301bdb7c55451e96793f78aec269ae; no voters: 8975ed20d415485ba322c4d9be7ef71d, a1f901ffff9e44949af805dd382c630f
I20260923 18:02:59.396878 31250 raft_consensus.cc:2726] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 18:02:59.474411 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:59.473900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:54165)
0923 18:02:59.474163 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:59.474226 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":90,"threads_started":1}
I20260923 18:02:59.606108 31252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:59.605684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:46835)
0923 18:02:59.605934 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:59.605995 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
I20260923 18:02:59.821658 31253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:02:59.821242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:37561)
0923 18:02:59.821501 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0923 18:02:59.821562 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":79,"threads_started":1}
I20260923 18:03:00.475986 31254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:00.475419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:34541)
0923 18:03:00.475816 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:00.475880 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":154,"threads_started":1}
I20260923 18:03:00.607395 31255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:00.606952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:40419)
0923 18:03:00.607234 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:00.607297 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":83,"threads_started":1}
I20260923 18:03:00.823030 31256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:00.822574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:55825)
0923 18:03:00.822871 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:00.822934 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":83,"threads_started":1}
I20260923 18:03:01.477157 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:01.476752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:34965)
0923 18:03:01.477009 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:01.477064 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":82,"threads_started":1}
I20260923 18:03:01.608544 31258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:01.608139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:34045)
0923 18:03:01.608394 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:01.608448 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260923 18:03:01.824654 31259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:01.824042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:37903)
0923 18:03:01.824471 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:01.824542 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":85,"threads_started":1}
I20260923 18:03:01.872609 31260 raft_consensus.cc:482] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:03:01.872821 31260 raft_consensus.cc:504] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:03:01.873209 31260 leader_election.cc:288] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579), a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369)
W20260923 18:03:01.874270 30854 leader_election.cc:341] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579): Not found: Tablet not found: 0fd8cd9512af4d55bcdb52da5ce532b7
W20260923 18:03:01.874498 30854 leader_election.cc:341] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369): Not found: Tablet not found: 0fd8cd9512af4d55bcdb52da5ce532b7
I20260923 18:03:01.874571 30854 leader_election.cc:302] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c301bdb7c55451e96793f78aec269ae; no voters: 8975ed20d415485ba322c4d9be7ef71d, a1f901ffff9e44949af805dd382c630f
I20260923 18:03:01.874794 31260 raft_consensus.cc:2726] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 18:03:02.478331 31261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:02.477911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:49203)
0923 18:03:02.478155 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:02.478212 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":75,"threads_started":1}
I20260923 18:03:02.609781 31262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:02.609341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:36005)
0923 18:03:02.609632 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:02.609687 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":86,"threads_started":1}
I20260923 18:03:02.825951 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:02.825484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:59709)
0923 18:03:02.825801 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:02.825856 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":70,"threads_started":1}
I20260923 18:03:03.479687 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:03.479148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:46301)
0923 18:03:03.479471 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:03.479586 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":101,"threads_started":1}
I20260923 18:03:03.611060 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:03.610642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:47713)
0923 18:03:03.610904 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:03.610964 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":78,"threads_started":1}
I20260923 18:03:03.827121 31266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:03.826715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:39953)
0923 18:03:03.826972 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:03.827028 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
I20260923 18:03:03.884284 31267 raft_consensus.cc:482] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:03:03.884433 31267 raft_consensus.cc:504] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:03:03.884850 31267 leader_election.cc:288] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579), a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369)
W20260923 18:03:03.885797 30854 leader_election.cc:341] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579): Not found: Tablet not found: 0fd8cd9512af4d55bcdb52da5ce532b7
W20260923 18:03:03.886001 30854 leader_election.cc:341] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369): Not found: Tablet not found: 0fd8cd9512af4d55bcdb52da5ce532b7
I20260923 18:03:03.886075 30854 leader_election.cc:302] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c301bdb7c55451e96793f78aec269ae; no voters: 8975ed20d415485ba322c4d9be7ef71d, a1f901ffff9e44949af805dd382c630f
I20260923 18:03:03.886294 31267 raft_consensus.cc:2726] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 18:03:04.480996 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:04.480546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:58347)
0923 18:03:04.480825 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:04.480901 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":112,"threads_started":1}
I20260923 18:03:04.612412 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:04.611983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:36707)
0923 18:03:04.612243 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:04.612308 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
I20260923 18:03:04.828430 31270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:04.827987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:45669)
0923 18:03:04.828244 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:04.828304 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":82,"threads_started":1}
I20260923 18:03:05.482642 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:05.482174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:57401)
0923 18:03:05.482441 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:05.482515 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":100,"threads_started":1}
I20260923 18:03:05.586529 31272 raft_consensus.cc:482] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:03:05.586668 31272 raft_consensus.cc:504] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8975ed20d415485ba322c4d9be7ef71d" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 35579 } } peers { permanent_uuid: "6c301bdb7c55451e96793f78aec269ae" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 36979 } } peers { permanent_uuid: "a1f901ffff9e44949af805dd382c630f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 40369 } }
I20260923 18:03:05.587059 31272 leader_election.cc:288] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579), a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369)
W20260923 18:03:05.588069 30854 leader_election.cc:341] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8975ed20d415485ba322c4d9be7ef71d (127.15.139.131:35579): Not found: Tablet not found: 0fd8cd9512af4d55bcdb52da5ce532b7
W20260923 18:03:05.588236 30854 leader_election.cc:341] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1f901ffff9e44949af805dd382c630f (127.15.139.130:40369): Not found: Tablet not found: 0fd8cd9512af4d55bcdb52da5ce532b7
I20260923 18:03:05.588302 30854 leader_election.cc:302] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c301bdb7c55451e96793f78aec269ae; no voters: 8975ed20d415485ba322c4d9be7ef71d, a1f901ffff9e44949af805dd382c630f
I20260923 18:03:05.588510 31272 raft_consensus.cc:2726] T 0fd8cd9512af4d55bcdb52da5ce532b7 P 6c301bdb7c55451e96793f78aec269ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 18:03:05.613806 31273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:05.613375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:41499)
0923 18:03:05.613656 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:05.613711 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":81,"threads_started":1}
I20260923 18:03:05.829761 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:05.829317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:41475)
0923 18:03:05.829603 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:05.829667 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":92,"threads_started":1}
I20260923 18:03:06.483928 31275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:06.483449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:54745)
0923 18:03:06.483763 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:06.483827 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":102,"threads_started":1}
I20260923 18:03:06.615160 31276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:06.614732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:39861)
0923 18:03:06.614999 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:06.615062 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":82,"threads_started":1}
I20260923 18:03:06.831029 31277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:06.830600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:53019)
0923 18:03:06.830862 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:06.830936 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":78,"threads_started":1}
I20260923 18:03:07.485292 31278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:07.484805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:43221)
0923 18:03:07.485107 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:07.485172 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":94,"threads_started":1}
I20260923 18:03:07.616467 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:07.616007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:40367)
0923 18:03:07.616283 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:07.616364 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":77,"threads_started":1}
I20260923 18:03:07.832398 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:07.831972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:35499)
0923 18:03:07.832237 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:07.832298 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":76,"threads_started":1}
I20260923 18:03:08.486694 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:08.486217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:37849)
0923 18:03:08.486528 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:08.486594 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":94,"threads_started":1}
I20260923 18:03:08.617892 31282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:08.617392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:55269)
0923 18:03:08.617720 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:08.617786 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":82,"threads_started":1}
I20260923 18:03:08.833694 31283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:08.833263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:57607)
0923 18:03:08.833536 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:08.833602 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":93,"threads_started":1}
I20260923 18:03:09.488044 31284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:09.487626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:60333)
0923 18:03:09.487879 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:09.487949 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260923 18:03:09.619272 31285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:09.618737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:48873)
0923 18:03:09.619049 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:09.619135 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":133,"threads_started":1}
I20260923 18:03:09.834978 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:09.834516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.131:44511)
0923 18:03:09.834808 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:09.834876 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":80,"threads_started":1}
I20260923 18:03:10.489578 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:10.488990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.129:42473)
0923 18:03:10.489410 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:10.489475 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":84,"threads_started":1}
I20260923 18:03:10.620754 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:10.620327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:44585 (local address 127.15.139.130:55047)
0923 18:03:10.620595 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:10.620654 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":78,"threads_started":1}
I20260923 18:03:10.815804 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 30803
I20260923 18:03:10.822034 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 30970
I20260923 18:03:10.827981 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 31104
2026-09-23T18:03:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19481 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:03:10 dist-test-slave-vr57 krb5kdc[31291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:03:10 dist-test-slave-vr57 krb5kdc[31291](info): set up 2 sockets
Sep 23 18:03:10 dist-test-slave-vr57 krb5kdc[31291](info): commencing operation
krb5kdc: starting...
W20260923 18:03:11.876312 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:03:11 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:03:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:03:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:03:12.003072 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36179
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:36179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:03:12.119705 31304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:03:12.120040 31304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:03:12.120121 31304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:03:12.124348 31304 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 18:03:12.124455 31304 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 18:03:12.124483 31304 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 18:03:12.124508 31304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:03:12.124580 31304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:03:12.124660 31304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:03:12.124680 31304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:03:12.124732 31304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:03:12.130461 31304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:36179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:03:12.131708 31304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:03:12.132807 31304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:03:12.142588 31309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:12.142598 31310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:12.142650 31312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:12.142588 31304 server_base.cc:1034] running on GCE node
I20260923 18:03:12.143546 31304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:03:12.144806 31304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:03:12.146020 31304 hybrid_clock.cc:648] HybridClock initialized: now 1790186592145986 us; error 48 us; skew 500 ppm
Sep 23 18:03:12 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186592, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:03:12.149194 31304 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:03:12.150168 31304 webserver.cc:460] Webserver started at http://127.15.139.190:33647/ using document root <none> and password file <none>
I20260923 18:03:12.150744 31304 fs_manager.cc:359] Metadata directory not provided
I20260923 18:03:12.150836 31304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:03:12.151132 31304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:03:12.153342 31304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "01dbfc16e7f74aa0b6047b3db1e030ec"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "5f01429844ed5a080ae428c8cf8bc0a4"
server_key_iv: "897cb93b1b0dfbc91b092064f3ff0faa"
server_key_version: "encryptionkey@0"
I20260923 18:03:12.153929 31304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "01dbfc16e7f74aa0b6047b3db1e030ec"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "5f01429844ed5a080ae428c8cf8bc0a4"
server_key_iv: "897cb93b1b0dfbc91b092064f3ff0faa"
server_key_version: "encryptionkey@0"
I20260923 18:03:12.157858 31304 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260923 18:03:12.161300 31319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:12.162621 31304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:03:12.162761 31304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "01dbfc16e7f74aa0b6047b3db1e030ec"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "5f01429844ed5a080ae428c8cf8bc0a4"
server_key_iv: "897cb93b1b0dfbc91b092064f3ff0faa"
server_key_version: "encryptionkey@0"
I20260923 18:03:12.162899 31304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:03:12.174409 31304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:03:12.177800 31304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:03:12.178027 31304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:03:12.185091 31304 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:36179
I20260923 18:03:12.185428 31371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:36179 every 8 connection(s)
I20260923 18:03:12.186092 31304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:03:12.188974 31372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:03:12.189723 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 31304
I20260923 18:03:12.189906 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:03:12.190249 15918 external_mini_cluster.cc:1428] Setting key 752b68b26ec7702220ce02e2e5a1ea8e
I20260923 18:03:12.195626 31372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Bootstrap starting.
I20260923 18:03:12.198663 31372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Neither blocks nor log segments found. Creating new log.
Sep 23 18:03:12 dist-test-slave-vr57 krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:03:12.199595 31372 log.cc:824] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Log is configured to *not* fsync() on all Append() calls
I20260923 18:03:12.201838 31372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: No bootstrap required, opened a new log
I20260923 18:03:12.204989 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:12.191704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42640 (local address 127.15.139.190:36179)
0923 18:03:12.192144 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:12.192159 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:12.192194 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:03:12.192844 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:12.192860 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:12.193187 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:12.193472 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:03:12.194368 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.195399 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.196614 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.197089 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.197215 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:12.200227 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:03:12.200232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:03:12.200246 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:03:12.200279 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:03:12.202054 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:12.202514 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:12.202520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:12.202524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:12.202636 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:12.202956 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:12.202959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:12.202961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:12.203359 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:03:12.203614 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:12.203912 (+ 298us) server_negotiation.cc:299] Negotiation successful
0923 18:03:12.204228 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":146,"threads_started":1}
I20260923 18:03:12.205770 31372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } }
I20260923 18:03:12.206028 31372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:03:12.206108 31372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01dbfc16e7f74aa0b6047b3db1e030ec, State: Initialized, Role: FOLLOWER
I20260923 18:03:12.206552 31372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } }
I20260923 18:03:12.206708 31372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:03:12.206796 31372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:03:12.206915 31372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:03:12.208025 31372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } }
I20260923 18:03:12.208385 31372 leader_election.cc:302] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01dbfc16e7f74aa0b6047b3db1e030ec; no voters:
I20260923 18:03:12.208711 31372 leader_election.cc:288] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:03:12.209038 31377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:03:12.209292 31377 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 1 LEADER]: Becoming Leader. State: Replica: 01dbfc16e7f74aa0b6047b3db1e030ec, State: Running, Role: LEADER
I20260923 18:03:12.209633 31377 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } }
I20260923 18:03:12.210140 31372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:03:12.211421 31379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01dbfc16e7f74aa0b6047b3db1e030ec. Latest consensus state: current_term: 1 leader_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } } }
I20260923 18:03:12.211592 31379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: This master's current role is: LEADER
I20260923 18:03:12.212018 31386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:03:12.211975 31378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } } }
I20260923 18:03:12.212128 31378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: This master's current role is: LEADER
I20260923 18:03:12.215009 31386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:03:12.220978 31386 catalog_manager.cc:1347] Generated new cluster ID: beb790a1cbb4499aa0542671d5fbde91
I20260923 18:03:12.221067 31386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:03:12.232435 31386 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:03:12.233875 31386 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:03:12.242524 31386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Generated new TSK 0
I20260923 18:03:12.243413 31386 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:03:12.278234 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36179
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:03:12.392083 31398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:03:12.392513 31398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:03:12.392603 31398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:03:12.398056 31398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:03:12.398137 31398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:03:12.398237 31398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:03:12.404076 31398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:03:12.405184 31398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:03:12.406255 31398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:03:12.416345 31404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:12.416364 31406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:12.416329 31403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:12.416553 31398 server_base.cc:1034] running on GCE node
I20260923 18:03:12.417011 31398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:03:12.417702 31398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:03:12.418883 31398 hybrid_clock.cc:648] HybridClock initialized: now 1790186592418870 us; error 37 us; skew 500 ppm
Sep 23 18:03:12 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186592, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:03:12.421682 31398 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:03:12.422595 31398 webserver.cc:460] Webserver started at http://127.15.139.129:43697/ using document root <none> and password file <none>
I20260923 18:03:12.423113 31398 fs_manager.cc:359] Metadata directory not provided
I20260923 18:03:12.423173 31398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:03:12.423350 31398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:03:12.425560 31398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "fdea49604a144312af4c718e7f4793b0"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "68bca8fee8b924f7e36a751972154feb"
server_key_iv: "a8da278e9a55da42f4c9466a97fc832e"
server_key_version: "encryptionkey@0"
I20260923 18:03:12.426087 31398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "fdea49604a144312af4c718e7f4793b0"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "68bca8fee8b924f7e36a751972154feb"
server_key_iv: "a8da278e9a55da42f4c9466a97fc832e"
server_key_version: "encryptionkey@0"
I20260923 18:03:12.429973 31398 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260923 18:03:12.432420 31413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:12.433673 31398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:03:12.433780 31398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "fdea49604a144312af4c718e7f4793b0"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "68bca8fee8b924f7e36a751972154feb"
server_key_iv: "a8da278e9a55da42f4c9466a97fc832e"
server_key_version: "encryptionkey@0"
I20260923 18:03:12.433918 31398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:03:12.469101 31398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:03:12.472595 31398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:03:12.472810 31398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:03:12.473443 31398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:03:12.474268 31398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:03:12.474313 31398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:12.474378 31398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:03:12.474416 31398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:12.484563 31398 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:35429
I20260923 18:03:12.484580 31526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:35429 every 8 connection(s)
I20260923 18:03:12.485529 31398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:03:12.495215 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 31398
I20260923 18:03:12.495383 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
Sep 23 18:03:12 dist-test-slave-vr57 krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186592, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:03:12.495826 15918 external_mini_cluster.cc:1428] Setting key 429682d4c2930eddc9405f33583f65c1
I20260923 18:03:12.500123 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:12.487622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:52701 (local address 127.15.139.190:36179)
0923 18:03:12.487816 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:12.487822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:12.488966 (+ 1144us) server_negotiation.cc:407] Connection header received
0923 18:03:12.489991 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:12.490002 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:12.490049 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:12.490129 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:03:12.491322 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.491945 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.493202 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.493436 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.493515 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:12.496246 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:03:12.496249 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:03:12.496251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:03:12.496276 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:03:12.497907 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:12.498443 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:12.498447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:12.498449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:12.498534 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:12.498894 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:12.498899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:12.498901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:12.499091 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:03:12.499210 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:12.499803 (+ 593us) server_negotiation.cc:299] Negotiation successful
0923 18:03:12.499955 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260923 18:03:12.501236 31529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:12.488105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:52701)
0923 18:03:12.488810 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:12.488852 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:12.489783 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:12.490346 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:12.490354 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:12.490709 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:03:12.491189 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:12.491201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.492085 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.492088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:12.493086 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:12.493096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.493578 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.493581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:12.493726 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:12.494361 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:03:12.494378 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:03:12.496068 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:03:12.498039 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:12.498047 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:12.498063 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:12.498335 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:12.498631 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:12.498635 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:12.498637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:12.498794 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:12.499219 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:03:12.499229 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:03:12.499494 (+ 265us) client_negotiation.cc:769] Sending connection context
0923 18:03:12.499748 (+ 254us) client_negotiation.cc:240] Negotiation successful
0923 18:03:12.500099 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":119,"threads_started":1}
I20260923 18:03:12.502642 31527 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36179
I20260923 18:03:12.502942 31527 heartbeater.cc:464] Registering TS with master...
I20260923 18:03:12.503626 31527 heartbeater.cc:511] Master 127.15.139.190:36179 requested a full tablet report, sending...
I20260923 18:03:12.505496 31336 ts_manager.cc:194] Registered new tserver with Master: fdea49604a144312af4c718e7f4793b0 (127.15.139.129:35429)
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
I20260923 18:03:12.507104 31336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:52701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:03:12.527370 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36179
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:03:12.643455 31532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:03:12.643811 31532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:03:12.643957 31532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:03:12.648248 31532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:03:12.648342 31532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:03:12.648434 31532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:03:12.653995 31532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:03:12.655124 31532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:03:12.656320 31532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:03:12.666119 31537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:12.666102 31538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:12.666103 31540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:12.666443 31532 server_base.cc:1034] running on GCE node
I20260923 18:03:12.666850 31532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:03:12.667608 31532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:03:12.668792 31532 hybrid_clock.cc:648] HybridClock initialized: now 1790186592668774 us; error 30 us; skew 500 ppm
Sep 23 18:03:12 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186592, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:03:12.671619 31532 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:03:12.672477 31532 webserver.cc:460] Webserver started at http://127.15.139.130:37607/ using document root <none> and password file <none>
I20260923 18:03:12.673069 31532 fs_manager.cc:359] Metadata directory not provided
I20260923 18:03:12.673157 31532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:03:12.673413 31532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:03:12.675434 31532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "e945f5ab43b3400aaa03cf626edaf194"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "a5d8ccd49eae7b3e629371518d29ad27"
server_key_iv: "2b649ffe75ea57bddf2cffd9947e9b65"
server_key_version: "encryptionkey@0"
I20260923 18:03:12.676051 31532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "e945f5ab43b3400aaa03cf626edaf194"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "a5d8ccd49eae7b3e629371518d29ad27"
server_key_iv: "2b649ffe75ea57bddf2cffd9947e9b65"
server_key_version: "encryptionkey@0"
I20260923 18:03:12.679792 31532 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:03:12.682315 31547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:12.683630 31532 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:03:12.683750 31532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "e945f5ab43b3400aaa03cf626edaf194"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "a5d8ccd49eae7b3e629371518d29ad27"
server_key_iv: "2b649ffe75ea57bddf2cffd9947e9b65"
server_key_version: "encryptionkey@0"
I20260923 18:03:12.683862 31532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:03:12.714397 31532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:03:12.717618 31532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:03:12.717828 31532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:03:12.718456 31532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:03:12.719312 31532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:03:12.719364 31532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:12.719440 31532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:03:12.719478 31532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:12.728654 31532 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:45995
I20260923 18:03:12.728674 31660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:45995 every 8 connection(s)
I20260923 18:03:12.729674 31532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:03:12.734606 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 31532
I20260923 18:03:12.734750 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:03:12.735028 15918 external_mini_cluster.cc:1428] Setting key 8ff2e6feb484511448b95b7ba703870d
Sep 23 18:03:12 dist-test-slave-vr57 krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186592, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:03:12.744900 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:12.731626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:36251 (local address 127.15.139.190:36179)
0923 18:03:12.731768 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:12.731772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:12.732874 (+ 1102us) server_negotiation.cc:407] Connection header received
0923 18:03:12.733883 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:12.733886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:12.733946 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:12.734053 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:03:12.735404 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.736357 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.737678 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.738114 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.738242 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:12.740988 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:03:12.740992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:03:12.740995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:03:12.741027 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:03:12.742608 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:12.743154 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:12.743157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:12.743159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:12.743243 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:12.743624 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:12.743629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:12.743630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:12.743821 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:03:12.743950 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:12.744593 (+ 643us) server_negotiation.cc:299] Negotiation successful
0923 18:03:12.744720 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
I20260923 18:03:12.745790 31663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:12.732034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:36251)
0923 18:03:12.732701 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:12.732745 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:12.733662 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:12.734243 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:12.734251 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:12.734631 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:03:12.735246 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:12.735269 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.736528 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.736532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:12.737544 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:12.737554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.738304 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.738309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:12.738501 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:12.739138 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:03:12.739155 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:03:12.740812 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:03:12.742754 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:12.742760 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:12.742776 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:12.743044 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:12.743339 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:12.743344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:12.743346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:12.743539 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:12.743950 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:03:12.743958 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:03:12.744286 (+ 328us) client_negotiation.cc:769] Sending connection context
0923 18:03:12.744534 (+ 248us) client_negotiation.cc:240] Negotiation successful
0923 18:03:12.744872 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":120,"threads_started":1}
I20260923 18:03:12.747016 31661 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36179
I20260923 18:03:12.747397 31661 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
I20260923 18:03:12.748066 31661 heartbeater.cc:511] Master 127.15.139.190:36179 requested a full tablet report, sending...
I20260923 18:03:12.749203 31336 ts_manager.cc:194] Registered new tserver with Master: e945f5ab43b3400aaa03cf626edaf194 (127.15.139.130:45995)
I20260923 18:03:12.750031 31336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:36251
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:03:12.764318 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36179
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:03:12.884292 31666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:03:12.884610 31666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:03:12.884708 31666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:03:12.889524 31666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:03:12.889600 31666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:03:12.889704 31666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:03:12.896131 31666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:03:12.897346 31666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:03:12.898509 31666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:03:12.909272 31672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:12.909358 31671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:12.909387 31666 server_base.cc:1034] running on GCE node
W20260923 18:03:12.909678 31674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:12.910219 31666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:03:12.911098 31666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:03:12.912318 31666 hybrid_clock.cc:648] HybridClock initialized: now 1790186592912290 us; error 34 us; skew 500 ppm
Sep 23 18:03:12 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186592, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:03:12.915493 31666 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:03:12.916451 31666 webserver.cc:460] Webserver started at http://127.15.139.131:35707/ using document root <none> and password file <none>
I20260923 18:03:12.917068 31666 fs_manager.cc:359] Metadata directory not provided
I20260923 18:03:12.917164 31666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:03:12.917389 31666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:03:12.919495 31666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "53ad0f661238425f8932049c81fc48b0"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "ad0815028e599ab4f2c150c4710c7ec1"
server_key_iv: "b34e4c9a6b69acb3145d5d56e9300ec1"
server_key_version: "encryptionkey@0"
I20260923 18:03:12.920198 31666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "53ad0f661238425f8932049c81fc48b0"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "ad0815028e599ab4f2c150c4710c7ec1"
server_key_iv: "b34e4c9a6b69acb3145d5d56e9300ec1"
server_key_version: "encryptionkey@0"
I20260923 18:03:12.924494 31666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260923 18:03:12.927079 31681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:12.928545 31666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:03:12.928696 31666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "53ad0f661238425f8932049c81fc48b0"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "ad0815028e599ab4f2c150c4710c7ec1"
server_key_iv: "b34e4c9a6b69acb3145d5d56e9300ec1"
server_key_version: "encryptionkey@0"
I20260923 18:03:12.928823 31666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:03:12.970443 31666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:03:12.974411 31666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:03:12.974680 31666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:03:12.975363 31666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:03:12.976351 31666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:03:12.976431 31666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:12.976517 31666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:03:12.976564 31666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:12.989584 31666 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:41273
I20260923 18:03:12.989602 31794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:41273 every 8 connection(s)
I20260923 18:03:12.990657 31666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:03:12.992039 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 31666
I20260923 18:03:12.992190 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:03:12.992499 15918 external_mini_cluster.cc:1428] Setting key 87223f28a473b09ed8eb7aee5b2654eb
Sep 23 18:03:12 dist-test-slave-vr57 krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186592, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:03:13.006654 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:12.993122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:60533 (local address 127.15.139.190:36179)
0923 18:03:12.993299 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:12.993303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:12.994606 (+ 1303us) server_negotiation.cc:407] Connection header received
0923 18:03:12.995492 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:12.995494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:12.995600 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:12.995686 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:03:12.997086 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.997619 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.998931 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.999140 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.999220 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.001940 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:03:13.001943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:03:13.001945 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:03:13.001970 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:03:13.003946 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:13.004487 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:13.004489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:13.004491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:13.004572 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:13.005085 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:13.005088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:13.005089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:13.005440 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:03:13.005585 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:13.006308 (+ 723us) server_negotiation.cc:299] Negotiation successful
0923 18:03:13.006480 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 18:03:13.007664 31797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:12.993661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:60533)
0923 18:03:12.994436 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:12.994491 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:12.995292 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:12.995867 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:12.995874 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:12.996253 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:03:12.996928 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:12.996943 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.997752 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.997756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:12.998812 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:12.998822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:12.999329 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:12.999332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:12.999477 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.000120 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:03:13.000137 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:03:13.001745 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:03:13.004083 (+ 2338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:13.004088 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:13.004103 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:13.004389 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:13.004684 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:13.004687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:13.004689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:13.004892 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:13.005599 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:03:13.005607 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:03:13.005993 (+ 386us) client_negotiation.cc:769] Sending connection context
0923 18:03:13.006232 (+ 239us) client_negotiation.cc:240] Negotiation successful
0923 18:03:13.006625 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":148,"threads_started":1}
I20260923 18:03:13.008773 31795 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36179
I20260923 18:03:13.009017 31795 heartbeater.cc:464] Registering TS with master...
I20260923 18:03:13.009517 31795 heartbeater.cc:511] Master 127.15.139.190:36179 requested a full tablet report, sending...
I20260923 18:03:13.010715 31336 ts_manager.cc:194] Registered new tserver with Master: 53ad0f661238425f8932049c81fc48b0 (127.15.139.131:41273)
I20260923 18:03:13.011688 31336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:60533
I20260923 18:03:13.018774 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:03:13.028949 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.020889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42648 (local address 127.15.139.190:36179)
0923 18:03:13.021034 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:13.021038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:13.021195 (+ 157us) server_negotiation.cc:407] Connection header received
0923 18:03:13.021292 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:13.021295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:13.021340 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:13.021420 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:03:13.022077 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.022637 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.023627 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.023845 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.023953 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.025361 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:03:13.025364 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:03:13.025366 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:03:13.025389 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:03:13.027194 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:13.027660 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:13.027663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:13.027664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:13.027768 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:13.028062 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:13.028068 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:13.028073 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:13.028374 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:03:13.028496 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:13.028722 (+ 226us) server_negotiation.cc:299] Negotiation successful
0923 18:03:13.028824 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260923 18:03:13.031791 31336 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:03:13.034955 31336 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:03:13.049950 31806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.043909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:41273 (local address 127.0.0.1:45048)
0923 18:03:13.044783 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:13.044834 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:13.044960 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:13.046095 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:13.046098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:13.046114 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:03:13.046209 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:13.046222 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.047414 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.047418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:13.048548 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:13.048558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.049378 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.049381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:13.049489 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.049525 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 18:03:13.049664 (+ 139us) client_negotiation.cc:240] Negotiation successful
0923 18:03:13.049810 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":40,"spinlock_wait_cycles":3712,"thread_start_us":90,"threads_started":1}
I20260923 18:03:13.049947 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.044032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45048 (local address 127.15.139.131:41273)
0923 18:03:13.045653 (+ 1621us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:13.045666 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:13.045692 (+ 26us) server_negotiation.cc:407] Connection header received
0923 18:03:13.045767 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:13.045772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:13.045950 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:13.046335 (+ 385us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:03:13.046470 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.047248 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.048661 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.049239 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.049388 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.049538 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:13.049695 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 18:03:13.049790 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1523,"thread_start_us":152,"threads_started":1}
I20260923 18:03:13.051162 31807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.043997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35429 (local address 127.0.0.1:55198)
0923 18:03:13.044785 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:13.044835 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:13.044962 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:13.045784 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:13.045791 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:13.045814 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:03:13.045965 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:13.045973 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.047534 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.047537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:13.049310 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:13.049317 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.050806 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.050809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:13.050893 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.050917 (+ 24us) client_negotiation.cc:769] Sending connection context
0923 18:03:13.050964 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 18:03:13.051022 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":20,"spinlock_wait_cycles":9856,"thread_start_us":112,"threads_started":1}
I20260923 18:03:13.052001 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.044888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55198 (local address 127.15.139.129:35429)
0923 18:03:13.045292 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:13.045300 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:13.045327 (+ 27us) server_negotiation.cc:407] Connection header received
0923 18:03:13.045407 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:13.045412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:13.045623 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:13.045783 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:03:13.046105 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.047384 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.049433 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.050661 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.051478 (+ 817us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.051667 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:13.051716 (+ 49us) server_negotiation.cc:299] Negotiation successful
0923 18:03:13.051849 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":81,"threads_started":1}
I20260923 18:03:13.052062 31809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.044254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:45995 (local address 127.0.0.1:52348)
0923 18:03:13.045160 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:13.045171 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:13.045231 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:13.045950 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:13.045954 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:13.045972 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:03:13.046085 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:13.046092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.047534 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.047537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:13.050499 (+ 2962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:13.050511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.051649 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.051653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:13.051736 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.051766 (+ 30us) client_negotiation.cc:769] Sending connection context
0923 18:03:13.051858 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 18:03:13.051928 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":3,"thread_start_us":108,"threads_started":1}
I20260923 18:03:13.052150 31811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.045203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52348 (local address 127.15.139.130:45995)
0923 18:03:13.045538 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:13.045547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:13.045565 (+ 18us) server_negotiation.cc:407] Connection header received
0923 18:03:13.045622 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:13.045626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:13.045815 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:13.045957 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:03:13.046201 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.047357 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.050646 (+ 3289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.051527 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.051636 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.051737 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:13.051895 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 18:03:13.051984 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":137,"threads_started":1}
I20260923 18:03:13.053407 31729 tablet_service.cc:1467] Processing CreateTablet for tablet 489ee12d06ff4ff8b3b4641a59dcc69e (DEFAULT_TABLE table=test-table [id=1447bcdfec9341188283547896d0ffbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:03:13.054492 31729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 489ee12d06ff4ff8b3b4641a59dcc69e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:03:13.054978 31461 tablet_service.cc:1467] Processing CreateTablet for tablet 489ee12d06ff4ff8b3b4641a59dcc69e (DEFAULT_TABLE table=test-table [id=1447bcdfec9341188283547896d0ffbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:03:13.056064 31461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 489ee12d06ff4ff8b3b4641a59dcc69e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:03:13.057473 31595 tablet_service.cc:1467] Processing CreateTablet for tablet 489ee12d06ff4ff8b3b4641a59dcc69e (DEFAULT_TABLE table=test-table [id=1447bcdfec9341188283547896d0ffbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:03:13.058451 31595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 489ee12d06ff4ff8b3b4641a59dcc69e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:03:13.061000 31812 tablet_bootstrap.cc:492] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0: Bootstrap starting.
I20260923 18:03:13.063201 31812 tablet_bootstrap.cc:654] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0: Neither blocks nor log segments found. Creating new log.
I20260923 18:03:13.063439 31813 tablet_bootstrap.cc:492] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Bootstrap starting.
I20260923 18:03:13.064425 31812 log.cc:824] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0: Log is configured to *not* fsync() on all Append() calls
I20260923 18:03:13.065930 31813 tablet_bootstrap.cc:654] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Neither blocks nor log segments found. Creating new log.
I20260923 18:03:13.066329 31815 tablet_bootstrap.cc:492] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Bootstrap starting.
I20260923 18:03:13.066817 31812 tablet_bootstrap.cc:492] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0: No bootstrap required, opened a new log
I20260923 18:03:13.066852 31813 log.cc:824] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Log is configured to *not* fsync() on all Append() calls
I20260923 18:03:13.067047 31812 ts_tablet_manager.cc:1397] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260923 18:03:13.069090 31813 tablet_bootstrap.cc:492] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: No bootstrap required, opened a new log
I20260923 18:03:13.069097 31815 tablet_bootstrap.cc:654] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Neither blocks nor log segments found. Creating new log.
I20260923 18:03:13.069303 31813 ts_tablet_manager.cc:1397] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260923 18:03:13.069968 31815 log.cc:824] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Log is configured to *not* fsync() on all Append() calls
I20260923 18:03:13.070535 31812 raft_consensus.cc:348] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:13.070832 31812 raft_consensus.cc:374] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:03:13.070900 31812 raft_consensus.cc:729] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ad0f661238425f8932049c81fc48b0, State: Initialized, Role: FOLLOWER
I20260923 18:03:13.071348 31812 consensus_queue.cc:260] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:13.072199 31815 tablet_bootstrap.cc:492] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: No bootstrap required, opened a new log
I20260923 18:03:13.072371 31812 ts_tablet_manager.cc:1428] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260923 18:03:13.072402 31815 ts_tablet_manager.cc:1397] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260923 18:03:13.072726 31795 heartbeater.cc:503] Master 127.15.139.190:36179 was elected leader, sending a full tablet report...
I20260923 18:03:13.073989 31813 raft_consensus.cc:348] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:13.074271 31813 raft_consensus.cc:374] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:03:13.074347 31813 raft_consensus.cc:729] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea49604a144312af4c718e7f4793b0, State: Initialized, Role: FOLLOWER
I20260923 18:03:13.074812 31813 consensus_queue.cc:260] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:13.075558 31527 heartbeater.cc:503] Master 127.15.139.190:36179 was elected leader, sending a full tablet report...
I20260923 18:03:13.075912 31813 ts_tablet_manager.cc:1428] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Time spent starting tablet: real 0.007s user 0.002s sys 0.006s
I20260923 18:03:13.077606 31815 raft_consensus.cc:348] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:13.077927 31815 raft_consensus.cc:374] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:03:13.077998 31815 raft_consensus.cc:729] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e945f5ab43b3400aaa03cf626edaf194, State: Initialized, Role: FOLLOWER
I20260923 18:03:13.078495 31815 consensus_queue.cc:260] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:13.079195 31661 heartbeater.cc:503] Master 127.15.139.190:36179 was elected leader, sending a full tablet report...
I20260923 18:03:13.079686 31815 ts_tablet_manager.cc:1428] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260923 18:03:13.230901 31662 tablet.cc:2369] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:03:13.237149 31528 tablet.cc:2369] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 18:03:13.241711 31796 tablet.cc:2369] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:03:13.473260 31819 raft_consensus.cc:482] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:03:13.473503 31819 raft_consensus.cc:504] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:13.474673 31819 leader_election.cc:288] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ad0f661238425f8932049c81fc48b0 (127.15.139.131:41273), e945f5ab43b3400aaa03cf626edaf194 (127.15.139.130:45995)
I20260923 18:03:13.480938 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.475107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:34183 (local address 127.15.139.131:41273)
0923 18:03:13.475250 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:13.475255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:13.475711 (+ 456us) server_negotiation.cc:407] Connection header received
0923 18:03:13.475826 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:13.475829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:13.475890 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:13.475993 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:03:13.476534 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.477419 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.479385 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.480101 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.480255 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.480387 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:13.480715 (+ 328us) server_negotiation.cc:299] Negotiation successful
0923 18:03:13.480801 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 18:03:13.480932 31811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.475104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:37689 (local address 127.15.139.130:45995)
0923 18:03:13.475238 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:13.475243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:13.475659 (+ 416us) server_negotiation.cc:407] Connection header received
0923 18:03:13.475833 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:13.475836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:13.475892 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:13.475994 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:03:13.476534 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.477421 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.479385 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.480136 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.480255 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.480313 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:13.480715 (+ 402us) server_negotiation.cc:299] Negotiation successful
0923 18:03:13.480801 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 18:03:13.480962 31822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.475022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:45995 (local address 127.15.139.129:37689)
0923 18:03:13.475490 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:13.475529 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:13.475693 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:13.476015 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:13.476018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:13.476043 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:03:13.476282 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:13.476288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.477539 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.477542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:13.479225 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:13.479237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.480258 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.480262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:13.480503 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.480564 (+ 61us) client_negotiation.cc:769] Sending connection context
0923 18:03:13.480644 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 18:03:13.480750 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":325,"spinlock_wait_cycles":20992,"thread_start_us":141,"threads_started":1}
I20260923 18:03:13.480963 31821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.475022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:41273 (local address 127.15.139.129:34183)
0923 18:03:13.475490 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:13.475529 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:13.475693 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:13.476015 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:13.476018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:13.476043 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:03:13.476274 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:13.476283 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.477538 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.477542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:13.479225 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:13.479237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.480258 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.480262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:13.480508 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.480558 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 18:03:13.480644 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 18:03:13.480750 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":374,"spinlock_wait_cycles":23936,"thread_start_us":99,"threads_started":1}
I20260923 18:03:13.481966 31615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "489ee12d06ff4ff8b3b4641a59dcc69e" candidate_uuid: "fdea49604a144312af4c718e7f4793b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e945f5ab43b3400aaa03cf626edaf194" is_pre_election: true
I20260923 18:03:13.481974 31749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "489ee12d06ff4ff8b3b4641a59dcc69e" candidate_uuid: "fdea49604a144312af4c718e7f4793b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ad0f661238425f8932049c81fc48b0" is_pre_election: true
I20260923 18:03:13.482280 31749 raft_consensus.cc:2454] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea49604a144312af4c718e7f4793b0 in term 0.
I20260923 18:03:13.482282 31615 raft_consensus.cc:2454] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea49604a144312af4c718e7f4793b0 in term 0.
I20260923 18:03:13.482769 31414 leader_election.cc:302] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 53ad0f661238425f8932049c81fc48b0, e945f5ab43b3400aaa03cf626edaf194, fdea49604a144312af4c718e7f4793b0; no voters:
I20260923 18:03:13.483038 31819 raft_consensus.cc:2781] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:03:13.483132 31819 raft_consensus.cc:482] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:03:13.483191 31819 raft_consensus.cc:3037] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:03:13.484301 31819 raft_consensus.cc:504] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:13.484859 31819 leader_election.cc:288] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [CANDIDATE]: Term 1 election: Requested vote from peers 53ad0f661238425f8932049c81fc48b0 (127.15.139.131:41273), e945f5ab43b3400aaa03cf626edaf194 (127.15.139.130:45995)
I20260923 18:03:13.485119 31615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "489ee12d06ff4ff8b3b4641a59dcc69e" candidate_uuid: "fdea49604a144312af4c718e7f4793b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e945f5ab43b3400aaa03cf626edaf194"
I20260923 18:03:13.485270 31749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "489ee12d06ff4ff8b3b4641a59dcc69e" candidate_uuid: "fdea49604a144312af4c718e7f4793b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ad0f661238425f8932049c81fc48b0"
I20260923 18:03:13.485270 31615 raft_consensus.cc:3037] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:03:13.485388 31749 raft_consensus.cc:3037] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:03:13.486622 31615 raft_consensus.cc:2454] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea49604a144312af4c718e7f4793b0 in term 1.
I20260923 18:03:13.486622 31749 raft_consensus.cc:2454] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea49604a144312af4c718e7f4793b0 in term 1.
I20260923 18:03:13.486948 31414 leader_election.cc:302] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e945f5ab43b3400aaa03cf626edaf194, fdea49604a144312af4c718e7f4793b0; no voters:
I20260923 18:03:13.487185 31819 raft_consensus.cc:2781] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:03:13.487440 31819 raft_consensus.cc:686] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 1 LEADER]: Becoming Leader. State: Replica: fdea49604a144312af4c718e7f4793b0, State: Running, Role: LEADER
I20260923 18:03:13.487788 31819 consensus_queue.cc:237] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:13.491138 31336 catalog_manager.cc:5495] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdea49604a144312af4c718e7f4793b0 (127.15.139.129). New cstate: current_term: 1 leader_uuid: "fdea49604a144312af4c718e7f4793b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } health_report { overall_health: UNKNOWN } } }
I20260923 18:03:13.611385 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.607561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55212 (local address 127.15.139.129:35429)
0923 18:03:13.607826 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:13.607830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:13.607844 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:03:13.607920 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:13.607922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:13.607980 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:13.608095 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:03:13.608320 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.608856 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.609834 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:13.610057 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:13.610138 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:13.610478 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:03:13.611032 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:03:13.611124 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:13.611215 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 18:03:13.611265 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":80,"threads_started":1}
I20260923 18:03:13.617411 31749 raft_consensus.cc:1261] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0 [term 1 FOLLOWER]: Refusing update from remote peer fdea49604a144312af4c718e7f4793b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:03:13.617448 31615 raft_consensus.cc:1261] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 1 FOLLOWER]: Refusing update from remote peer fdea49604a144312af4c718e7f4793b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:03:13.618172 31823 consensus_queue.cc:1035] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:03:13.618358 31819 consensus_queue.cc:1035] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:03:13.628612 31832 mvcc.cc:205] Tried to move back new op lower bound from 7332604287449387008 to 7332604286928855040. Current Snapshot: MvccSnapshot[applied={T|T < 7332604287449387008}]
I20260923 18:03:13.633220 31336 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:03:13.633456 31336 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:03:13.636180 31336 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Sending DeleteTablet for 3 replicas of tablet 489ee12d06ff4ff8b3b4641a59dcc69e
I20260923 18:03:13.637022 31729 tablet_service.cc:1514] Processing DeleteTablet for tablet 489ee12d06ff4ff8b3b4641a59dcc69e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:03:13 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:45048
I20260923 18:03:13.637048 31461 tablet_service.cc:1514] Processing DeleteTablet for tablet 489ee12d06ff4ff8b3b4641a59dcc69e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:03:13 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:55198
I20260923 18:03:13.637245 31595 tablet_service.cc:1514] Processing DeleteTablet for tablet 489ee12d06ff4ff8b3b4641a59dcc69e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:03:13 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:52348
I20260923 18:03:13.637547 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 31398
I20260923 18:03:13.639245 31840 tablet_replica.cc:330] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0: stopping tablet replica
I20260923 18:03:13.639606 31840 raft_consensus.cc:2229] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:03:13.639919 31840 raft_consensus.cc:2258] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:03:13.641407 31840 ts_tablet_manager.cc:1905] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:03:13.643765 31842 tablet_replica.cc:330] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: stopping tablet replica
I20260923 18:03:13.644096 31842 raft_consensus.cc:2229] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:03:13.644446 31842 raft_consensus.cc:2258] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:03:13.645064 31840 ts_tablet_manager.cc:1918] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:03:13.645174 31840 log.cc:1196] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/489ee12d06ff4ff8b3b4641a59dcc69e
I20260923 18:03:13.645502 31840 ts_tablet_manager.cc:1939] T 489ee12d06ff4ff8b3b4641a59dcc69e P 53ad0f661238425f8932049c81fc48b0: Deleting consensus metadata
I20260923 18:03:13.645877 31842 ts_tablet_manager.cc:1905] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 18:03:13.646391 31323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:35429 (error 108)
I20260923 18:03:13.646608 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 31532
I20260923 18:03:13.648344 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.647862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35429 (local address 127.0.0.1:55226)
0923 18:03:13.648122 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:13.648210 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":109,"threads_started":1}
I20260923 18:03:13.648481 31321 catalog_manager.cc:4841] TS 53ad0f661238425f8932049c81fc48b0 (127.15.139.131:41273): tablet 489ee12d06ff4ff8b3b4641a59dcc69e (table test-table [id=1447bcdfec9341188283547896d0ffbd]) successfully deleted
W20260923 18:03:13.648535 31323 catalog_manager.cc:4584] TS fdea49604a144312af4c718e7f4793b0 (127.15.139.129:35429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 489ee12d06ff4ff8b3b4641a59dcc69e: Network error: Client connection negotiation failed: client connection to 127.15.139.129:35429: connect: Connection refused (error 111)
I20260923 18:03:13.652842 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 31666
I20260923 18:03:13.653306 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:13.652987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:45995 (local address 127.0.0.1:52362)
0923 18:03:13.653165 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:13.653217 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.130:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260923 18:03:13.658633 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 31304
I20260923 18:03:33.665597 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36179
--webserver_interface=127.15.139.190
--webserver_port=33647
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:36179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:03:33.776376 31845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:03:33.776713 31845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:03:33.776785 31845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:03:33.780870 31845 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 18:03:33.780949 31845 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 18:03:33.780977 31845 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 18:03:33.781008 31845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:03:33.781050 31845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:03:33.781095 31845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:03:33.781129 31845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:03:33.781159 31845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:03:33.786653 31845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:36179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=33647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:03:33.787793 31845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:03:33.788872 31845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 18:03:33.803133 31845 server_base.cc:1034] running on GCE node
W20260923 18:03:33.803188 31854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:33.803121 31851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:33.803104 31852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:33.804114 31845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:03:33.805356 31845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:03:33.806560 31845 hybrid_clock.cc:648] HybridClock initialized: now 1790186613806542 us; error 35 us; skew 500 ppm
Sep 23 18:03:33 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:03:33.810138 31845 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:03:33.811264 31845 webserver.cc:460] Webserver started at http://127.15.139.190:33647/ using document root <none> and password file <none>
I20260923 18:03:33.811985 31845 fs_manager.cc:359] Metadata directory not provided
I20260923 18:03:33.812103 31845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:03:33.816247 31845 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:03:33.818591 31861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:33.820099 31845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:03:33.820241 31845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "01dbfc16e7f74aa0b6047b3db1e030ec"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "5f01429844ed5a080ae428c8cf8bc0a4"
server_key_iv: "897cb93b1b0dfbc91b092064f3ff0faa"
server_key_version: "encryptionkey@0"
I20260923 18:03:33.820904 31845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:03:33.838004 31845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:03:33.841190 31845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:03:33.841405 31845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:03:33.848912 31845 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:36179
I20260923 18:03:33.849246 31913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:36179 every 8 connection(s)
I20260923 18:03:33.850116 31845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:03:33.852515 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 31845
I20260923 18:03:33.852954 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:35429
--local_ip_for_outbound_sockets=127.15.139.129
--tserver_master_addrs=127.15.139.190:36179
--webserver_port=43697
--webserver_interface=127.15.139.129
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:03:33.854980 31914 sys_catalog.cc:263] Verifying existing consensus state
I20260923 18:03:33.856848 31914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Bootstrap starting.
I20260923 18:03:33.874604 31914 log.cc:824] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Log is configured to *not* fsync() on all Append() calls
I20260923 18:03:33.883590 31914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 18:03:33.883970 31914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Bootstrap complete.
I20260923 18:03:33.888383 31914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } }
I20260923 18:03:33.888823 31914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01dbfc16e7f74aa0b6047b3db1e030ec, State: Initialized, Role: FOLLOWER
I20260923 18:03:33.889325 31914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } }
I20260923 18:03:33.889479 31914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:03:33.889578 31914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:03:33.889698 31914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 1 FOLLOWER]: Advancing to term 2
I20260923 18:03:33.892742 31914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } }
I20260923 18:03:33.893172 31914 leader_election.cc:302] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01dbfc16e7f74aa0b6047b3db1e030ec; no voters:
I20260923 18:03:33.893605 31914 leader_election.cc:288] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 18:03:33.893697 31918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 2 FOLLOWER]: Leader election won for term 2
I20260923 18:03:33.893913 31918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 2 LEADER]: Becoming Leader. State: Replica: 01dbfc16e7f74aa0b6047b3db1e030ec, State: Running, Role: LEADER
I20260923 18:03:33.894225 31918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } }
I20260923 18:03:33.894912 31914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:03:33.895222 31918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01dbfc16e7f74aa0b6047b3db1e030ec. Latest consensus state: current_term: 2 leader_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } } }
I20260923 18:03:33.895347 31918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: This master's current role is: LEADER
I20260923 18:03:33.895790 31919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } } }
I20260923 18:03:33.895987 31919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: This master's current role is: LEADER
I20260923 18:03:33.900403 31933 catalog_manager.cc:1259] Loaded cluster ID: beb790a1cbb4499aa0542671d5fbde91
I20260923 18:03:33.900465 31933 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: loading cluster ID for follower catalog manager: success
I20260923 18:03:33.903131 31933 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: acquiring CA information for follower catalog manager: success
I20260923 18:03:33.904764 31933 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 18:03:33.904901 31922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:03:33.907348 31922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:03:33.907891 31922 catalog_manager.cc:1259] Loaded cluster ID: beb790a1cbb4499aa0542671d5fbde91
I20260923 18:03:33.907953 31922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:03:33.908769 31922 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:03:33.909240 31922 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Loaded TSK: 0
I20260923 18:03:33.934049 31922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Generated new TSK 1
I20260923 18:03:33.934360 31922 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 18:03:33.972746 31916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:03:33.973057 31916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:03:33.973140 31916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:03:33.977341 31916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:03:33.977440 31916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:03:33.977543 31916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:03:33.983095 31916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:35429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=43697
--tserver_master_addrs=127.15.139.190:36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:03:33.984405 31916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:03:33.985556 31916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:03:33.996516 31942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:33.996562 31941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:33.996623 31944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:33.997555 31916 server_base.cc:1034] running on GCE node
I20260923 18:03:33.998001 31916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:03:33.998787 31916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:03:33.999990 31916 hybrid_clock.cc:648] HybridClock initialized: now 1790186613999974 us; error 48 us; skew 500 ppm
Sep 23 18:03:33 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:03:34.007261 31916 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:03:34.008451 31916 webserver.cc:460] Webserver started at http://127.15.139.129:43697/ using document root <none> and password file <none>
I20260923 18:03:34.009155 31916 fs_manager.cc:359] Metadata directory not provided
I20260923 18:03:34.009258 31916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:03:34.014452 31916 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260923 18:03:34.017127 31951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:34.018631 31916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:03:34.018761 31916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "fdea49604a144312af4c718e7f4793b0"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "68bca8fee8b924f7e36a751972154feb"
server_key_iv: "a8da278e9a55da42f4c9466a97fc832e"
server_key_version: "encryptionkey@0"
I20260923 18:03:34.019526 31916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:03:34.046437 31916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:03:34.050360 31916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:03:34.050549 31916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:03:34.051220 31916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:03:34.052567 31958 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 18:03:34.056667 31916 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 18:03:34.056741 31916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260923 18:03:34.056806 31916 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 18:03:34.059099 31916 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 18:03:34.059154 31916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260923 18:03:34.059561 31958 tablet_bootstrap.cc:492] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Bootstrap starting.
I20260923 18:03:34.069100 31916 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:35429
I20260923 18:03:34.069494 32065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:35429 every 8 connection(s)
I20260923 18:03:34.070241 31916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:03:34.080482 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 31916
I20260923 18:03:34.080785 31958 log.cc:824] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Log is configured to *not* fsync() on all Append() calls
I20260923 18:03:34.081004 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:45995
--local_ip_for_outbound_sockets=127.15.139.130
--tserver_master_addrs=127.15.139.190:36179
--webserver_port=37607
--webserver_interface=127.15.139.130
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 18:03:34 dist-test-slave-vr57 krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:03:34.089263 31958 tablet_bootstrap.cc:492] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 18:03:34.089699 31958 tablet_bootstrap.cc:492] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Bootstrap complete.
I20260923 18:03:34.090003 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.072530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:47527 (local address 127.15.139.190:36179)
0923 18:03:34.072875 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:34.072894 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:34.073363 (+ 469us) server_negotiation.cc:407] Connection header received
0923 18:03:34.075009 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:34.075020 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:34.075317 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:34.075714 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:03:34.077277 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.078257 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.079897 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.080343 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.080521 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.083792 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:03:34.083797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:03:34.083816 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:03:34.083843 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:03:34.086006 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.086695 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.086702 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.086709 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.086864 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.087320 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.087324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.087325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.087786 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:03:34.087916 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:34.088663 (+ 747us) server_negotiation.cc:299] Negotiation successful
0923 18:03:34.088954 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":120,"threads_started":1}
I20260923 18:03:34.090031 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.072504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:47527)
0923 18:03:34.073188 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:34.073243 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:34.074557 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:34.075777 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:34.075786 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:34.076290 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:03:34.077060 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.077090 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.078428 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.078432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.079778 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.079788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.080506 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.080510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.080699 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.081545 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:03:34.081566 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:03:34.083536 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:03:34.086138 (+ 2602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:34.086146 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:34.086162 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:34.086563 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:34.086978 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:34.086984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:34.086990 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:34.087203 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:34.087917 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:03:34.087926 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:03:34.088252 (+ 326us) client_negotiation.cc:769] Sending connection context
0923 18:03:34.088494 (+ 242us) client_negotiation.cc:240] Negotiation successful
0923 18:03:34.088859 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":130,"threads_started":1}
I20260923 18:03:34.091883 31958 ts_tablet_manager.cc:1397] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Time spent bootstrapping tablet: real 0.033s user 0.020s sys 0.010s
I20260923 18:03:34.092700 32066 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36179
I20260923 18:03:34.093004 32066 heartbeater.cc:464] Registering TS with master...
I20260923 18:03:34.093613 32066 heartbeater.cc:511] Master 127.15.139.190:36179 requested a full tablet report, sending...
I20260923 18:03:34.094547 31958 raft_consensus.cc:348] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:34.094990 31958 raft_consensus.cc:729] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea49604a144312af4c718e7f4793b0, State: Initialized, Role: FOLLOWER
I20260923 18:03:34.095467 31958 consensus_queue.cc:260] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:34.096514 31958 ts_tablet_manager.cc:1428] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260923 18:03:34.096726 31878 ts_manager.cc:194] Registered new tserver with Master: fdea49604a144312af4c718e7f4793b0 (127.15.139.129:35429)
I20260923 18:03:34.100660 31878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:47527
I20260923 18:03:34.103459 32066 heartbeater.cc:503] Master 127.15.139.190:36179 was elected leader, sending a full tablet report...
Sep 23 18:03:34 dist-test-slave-vr57 krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.129@KRBTEST.COM
I20260923 18:03:34.111632 32073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.100016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35429 (local address 127.0.0.1:55958)
0923 18:03:34.100622 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:34.100672 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:34.100832 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:34.101531 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:34.101537 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:34.101968 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:03:34.102102 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.102109 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.103442 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.103446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.104465 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.104476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.105245 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.105250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.105369 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.105958 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:03:34.105972 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:03:34.107779 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:03:34.109832 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:34.109841 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:34.109847 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:34.110147 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:34.110431 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:34.110434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:34.110436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:34.110533 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:34.110964 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:03:34.110970 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:03:34.111093 (+ 123us) client_negotiation.cc:769] Sending connection context
0923 18:03:34.111237 (+ 144us) client_negotiation.cc:240] Negotiation successful
0923 18:03:34.111428 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":115,"threads_started":1}
I20260923 18:03:34.111644 32074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.100069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55958 (local address 127.15.139.129:35429)
0923 18:03:34.101122 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:34.101132 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:34.101151 (+ 19us) server_negotiation.cc:407] Connection header received
0923 18:03:34.101208 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:34.101212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:34.101373 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:34.101548 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:03:34.102240 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.103303 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.104603 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.105126 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.105407 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.107940 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:03:34.107950 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:03:34.107960 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:03:34.107983 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:03:34.109717 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.110260 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.110264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.110265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.110341 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.110622 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.110624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.110625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.110853 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:03:34.110954 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:34.111310 (+ 356us) server_negotiation.cc:299] Negotiation successful
0923 18:03:34.111455 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":970,"thread_start_us":95,"threads_started":1}
I20260923 18:03:34.112808 32000 tablet_service.cc:1514] Processing DeleteTablet for tablet 489ee12d06ff4ff8b3b4641a59dcc69e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:03:13 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:55958
I20260923 18:03:34.114039 32075 tablet_replica.cc:330] stopping tablet replica
I20260923 18:03:34.114359 32075 raft_consensus.cc:2229] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:03:34.114626 32075 raft_consensus.cc:2258] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:03:34.115751 32075 ts_tablet_manager.cc:1905] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:03:34.120251 32075 ts_tablet_manager.cc:1918] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:03:34.120357 32075 log.cc:1196] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/489ee12d06ff4ff8b3b4641a59dcc69e
I20260923 18:03:34.120637 32075 ts_tablet_manager.cc:1939] T 489ee12d06ff4ff8b3b4641a59dcc69e P fdea49604a144312af4c718e7f4793b0: Deleting consensus metadata
I20260923 18:03:34.122725 31865 catalog_manager.cc:4841] TS fdea49604a144312af4c718e7f4793b0 (127.15.139.129:35429): tablet 489ee12d06ff4ff8b3b4641a59dcc69e (table test-table [id=1447bcdfec9341188283547896d0ffbd]) successfully deleted
W20260923 18:03:34.204782 32071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:03:34.205044 32071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:03:34.205087 32071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:03:34.209161 32071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:03:34.209221 32071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:03:34.209278 32071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:03:34.214442 32071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:45995
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=37607
--tserver_master_addrs=127.15.139.190:36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:03:34.215392 32071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:03:34.216347 32071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:03:34.225979 32081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:34.226018 32083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:34.225953 32080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:34.226320 32071 server_base.cc:1034] running on GCE node
I20260923 18:03:34.226712 32071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:03:34.227360 32071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:03:34.228534 32071 hybrid_clock.cc:648] HybridClock initialized: now 1790186614228514 us; error 39 us; skew 500 ppm
Sep 23 18:03:34 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186614, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:03:34.231367 32071 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:03:34.232311 32071 webserver.cc:460] Webserver started at http://127.15.139.130:37607/ using document root <none> and password file <none>
I20260923 18:03:34.232843 32071 fs_manager.cc:359] Metadata directory not provided
I20260923 18:03:34.232918 32071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:03:34.236848 32071 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260923 18:03:34.239027 32090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:34.240532 32071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:03:34.240669 32071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "e945f5ab43b3400aaa03cf626edaf194"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "a5d8ccd49eae7b3e629371518d29ad27"
server_key_iv: "2b649ffe75ea57bddf2cffd9947e9b65"
server_key_version: "encryptionkey@0"
I20260923 18:03:34.241214 32071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:03:34.255627 32071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:03:34.258643 32071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:03:34.258865 32071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:03:34.259474 32071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:03:34.260646 32097 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 18:03:34.264768 32071 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 18:03:34.264858 32071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260923 18:03:34.264967 32071 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 18:03:34.266636 32071 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 18:03:34.266706 32071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260923 18:03:34.266973 32097 tablet_bootstrap.cc:492] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Bootstrap starting.
I20260923 18:03:34.272413 32097 log.cc:824] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Log is configured to *not* fsync() on all Append() calls
I20260923 18:03:34.277450 32071 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:45995
I20260923 18:03:34.277508 32205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:45995 every 8 connection(s)
I20260923 18:03:34.278357 32071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:03:34.280200 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 32071
I20260923 18:03:34.280637 32097 tablet_bootstrap.cc:492] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 18:03:34.280687 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:41273
--local_ip_for_outbound_sockets=127.15.139.131
--tserver_master_addrs=127.15.139.190:36179
--webserver_port=35707
--webserver_interface=127.15.139.131
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:03:34.281082 32097 tablet_bootstrap.cc:492] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Bootstrap complete.
I20260923 18:03:34.281447 32097 ts_tablet_manager.cc:1397] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260923 18:03:34.284395 32097 raft_consensus.cc:348] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:34.285049 32097 raft_consensus.cc:729] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e945f5ab43b3400aaa03cf626edaf194, State: Initialized, Role: FOLLOWER
I20260923 18:03:34.285684 32097 consensus_queue.cc:260] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } }
I20260923 18:03:34.287005 32097 ts_tablet_manager.cc:1428] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
Sep 23 18:03:34 dist-test-slave-vr57 krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186614, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:03:34.293107 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.280203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:60675 (local address 127.15.139.190:36179)
0923 18:03:34.280427 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:34.280432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:34.281294 (+ 862us) server_negotiation.cc:407] Connection header received
0923 18:03:34.282399 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:34.282403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:34.282468 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:34.282573 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:03:34.284185 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.284726 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.286199 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.286512 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.286577 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.289542 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:03:34.289546 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:03:34.289549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:03:34.289582 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:03:34.291008 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.291557 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.291561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.291563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.291648 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.291995 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.292000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.292002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.292201 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:03:34.292311 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:34.292792 (+ 481us) server_negotiation.cc:299] Negotiation successful
0923 18:03:34.292933 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260923 18:03:34.293942 32208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.280357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:60675)
0923 18:03:34.281090 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:34.281189 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:34.282132 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:34.282768 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:34.282780 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:34.283322 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:03:34.284016 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.284039 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.284889 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.284894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.286027 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.286037 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.286711 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.286713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.286853 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.287894 (+ 1041us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:03:34.287911 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:03:34.289366 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:03:34.291129 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:34.291135 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:34.291149 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:34.291416 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:34.291745 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:34.291749 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:34.291751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:34.291896 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:34.292309 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:03:34.292317 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:03:34.292558 (+ 241us) client_negotiation.cc:769] Sending connection context
0923 18:03:34.292749 (+ 191us) client_negotiation.cc:240] Negotiation successful
0923 18:03:34.293079 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":155,"threads_started":1}
I20260923 18:03:34.295246 32206 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36179
I20260923 18:03:34.295531 32206 heartbeater.cc:464] Registering TS with master...
I20260923 18:03:34.296149 32206 heartbeater.cc:511] Master 127.15.139.190:36179 requested a full tablet report, sending...
I20260923 18:03:34.297894 31878 ts_manager.cc:194] Registered new tserver with Master: e945f5ab43b3400aaa03cf626edaf194 (127.15.139.130:45995)
I20260923 18:03:34.299172 31878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:60675
I20260923 18:03:34.301472 32206 heartbeater.cc:503] Master 127.15.139.190:36179 was elected leader, sending a full tablet report...
Sep 23 18:03:34 dist-test-slave-vr57 krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.130@KRBTEST.COM
I20260923 18:03:34.308816 32073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.298813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:45995 (local address 127.0.0.1:51706)
0923 18:03:34.298976 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:34.298990 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:34.299071 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:34.299822 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:34.299826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:34.300022 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:03:34.300123 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.300130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.301460 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.301464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.302357 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.302368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.303116 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.303120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.303227 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.303775 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:03:34.303789 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:03:34.305149 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:03:34.307114 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:34.307119 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:34.307121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:34.307332 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:34.307700 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:34.307703 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:34.307705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:34.307796 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:34.308255 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:03:34.308258 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:03:34.308358 (+ 100us) client_negotiation.cc:769] Sending connection context
0923 18:03:34.308481 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 18:03:34.308639 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260923 18:03:34.308898 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.298948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51706 (local address 127.15.139.130:45995)
0923 18:03:34.299303 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:34.299315 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:34.299336 (+ 21us) server_negotiation.cc:407] Connection header received
0923 18:03:34.299399 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:34.299405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:34.299670 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:34.299808 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:03:34.300248 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.301328 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.302487 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.303007 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.303092 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.305294 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:03:34.305303 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:03:34.305313 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:03:34.305334 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:03:34.306976 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.307451 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.307462 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.307463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.307602 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.307892 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.307896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.307898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.308146 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:03:34.308292 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:34.308572 (+ 280us) server_negotiation.cc:299] Negotiation successful
0923 18:03:34.308735 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":117,"threads_started":1}
I20260923 18:03:34.310292 32139 tablet_service.cc:1514] Processing DeleteTablet for tablet 489ee12d06ff4ff8b3b4641a59dcc69e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:03:13 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:51706
I20260923 18:03:34.311205 32212 tablet_replica.cc:330] stopping tablet replica
I20260923 18:03:34.311553 32212 raft_consensus.cc:2229] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:03:34.311827 32212 raft_consensus.cc:2258] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:03:34.312789 32212 ts_tablet_manager.cc:1905] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:03:34.316540 32212 ts_tablet_manager.cc:1918] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:03:34.316624 32212 log.cc:1196] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/489ee12d06ff4ff8b3b4641a59dcc69e
I20260923 18:03:34.316901 32212 ts_tablet_manager.cc:1939] T 489ee12d06ff4ff8b3b4641a59dcc69e P e945f5ab43b3400aaa03cf626edaf194: Deleting consensus metadata
I20260923 18:03:34.318499 31862 catalog_manager.cc:4841] TS e945f5ab43b3400aaa03cf626edaf194 (127.15.139.130:45995): tablet 489ee12d06ff4ff8b3b4641a59dcc69e (table test-table [id=1447bcdfec9341188283547896d0ffbd]) successfully deleted
W20260923 18:03:34.401645 32209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:03:34.401908 32209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:03:34.401952 32209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:03:34.405951 32209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:03:34.406008 32209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:03:34.406066 32209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:03:34.411561 32209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:41273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=35707
--tserver_master_addrs=127.15.139.190:36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:03:34.412652 32209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:03:34.413676 32209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:03:34.423327 32218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:34.423378 32217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:34.423714 32209 server_base.cc:1034] running on GCE node
W20260923 18:03:34.423341 32220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:34.424259 32209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:03:34.424963 32209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:03:34.426148 32209 hybrid_clock.cc:648] HybridClock initialized: now 1790186614426107 us; error 46 us; skew 500 ppm
Sep 23 18:03:34 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186614, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:03:34.429008 32209 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:03:34.429893 32209 webserver.cc:460] Webserver started at http://127.15.139.131:35707/ using document root <none> and password file <none>
I20260923 18:03:34.430459 32209 fs_manager.cc:359] Metadata directory not provided
I20260923 18:03:34.430529 32209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:03:34.434505 32209 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:03:34.436681 32227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:34.437953 32209 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:03:34.438073 32209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "53ad0f661238425f8932049c81fc48b0"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "ad0815028e599ab4f2c150c4710c7ec1"
server_key_iv: "b34e4c9a6b69acb3145d5d56e9300ec1"
server_key_version: "encryptionkey@0"
I20260923 18:03:34.438628 32209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:03:34.455374 32209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:03:34.458834 32209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:03:34.459077 32209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:03:34.459843 32209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:03:34.460777 32209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:03:34.460835 32209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:34.460937 32209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:03:34.460986 32209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:34.471916 32209 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:41273
I20260923 18:03:34.471936 32340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:41273 every 8 connection(s)
I20260923 18:03:34.473165 32209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:03:34.478274 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 32209
Sep 23 18:03:34 dist-test-slave-vr57 krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186614, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:03:34.487437 32345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.479008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40722 (local address 127.15.139.190:36179)
0923 18:03:34.479238 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:34.479242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:34.479370 (+ 128us) server_negotiation.cc:407] Connection header received
0923 18:03:34.479470 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:34.479473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:34.479541 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:34.479636 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:03:34.480515 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.481015 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.482145 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.482429 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.482541 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.483963 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:03:34.483967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:03:34.483970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:03:34.484000 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:03:34.485730 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.486153 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.486157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.486159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.486241 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.486527 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.486530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.486532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.486758 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:03:34.486974 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:34.487131 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 18:03:34.487253 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":74,"threads_started":1}
I20260923 18:03:34.488091 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.475132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:42425 (local address 127.15.139.190:36179)
0923 18:03:34.475271 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:34.475274 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:34.476388 (+ 1114us) server_negotiation.cc:407] Connection header received
0923 18:03:34.477329 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:34.477332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:34.477394 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:34.477502 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:03:34.478796 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.479638 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.481117 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.481338 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.481412 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.484243 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:03:34.484247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:03:34.484249 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:03:34.484310 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:03:34.485875 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.486438 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.486443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.486445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.486526 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.486862 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.486867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.486869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.487052 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:03:34.487158 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:34.487795 (+ 637us) server_negotiation.cc:299] Negotiation successful
0923 18:03:34.487903 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 18:03:34.489022 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.475564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:42425)
0923 18:03:34.476231 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:34.476272 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:34.477104 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:34.477680 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:34.477687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:34.478071 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:03:34.478619 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.478634 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.479765 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.479770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.480998 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.481008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.481483 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.481488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.481681 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.482359 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:03:34.482376 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:03:34.484082 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:03:34.486058 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:34.486063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:34.486078 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:34.486333 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:34.486618 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:03:34.486622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:03:34.486625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:03:34.486763 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:03:34.487159 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:03:34.487167 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:03:34.487489 (+ 322us) client_negotiation.cc:769] Sending connection context
0923 18:03:34.487734 (+ 245us) client_negotiation.cc:240] Negotiation successful
0923 18:03:34.488083 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":115,"threads_started":1}
I20260923 18:03:34.490379 32341 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36179
I20260923 18:03:34.490629 32341 heartbeater.cc:464] Registering TS with master...
I20260923 18:03:34.491094 32341 heartbeater.cc:511] Master 127.15.139.190:36179 requested a full tablet report, sending...
I20260923 18:03:34.492062 31878 ts_manager.cc:194] Registered new tserver with Master: 53ad0f661238425f8932049c81fc48b0 (127.15.139.131:41273)
I20260923 18:03:34.492612 31878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:42425
I20260923 18:03:34.501251 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:03:34.505565 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.501944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40724 (local address 127.15.139.190:36179)
0923 18:03:34.502072 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:34.502075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:34.502212 (+ 137us) server_negotiation.cc:407] Connection header received
0923 18:03:34.502354 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:34.502356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:34.502454 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:34.502544 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:03:34.502808 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.503313 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.504401 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.504631 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.504725 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.505111 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:03:34.505246 (+ 135us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0923 18:03:34.505468 (+ 222us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40724: token expired
Metrics: {"server-negotiator.queue_time_us":49}
W20260923 18:03:34.505656 32068 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40724: token expired
W20260923 18:03:34.506222 31801 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.139.190:36179)
I20260923 18:03:34.513537 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.506582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.15.139.190:36179)
0923 18:03:34.506705 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:34.506708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:34.506722 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:03:34.506822 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:34.506824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:34.506862 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:34.506933 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:03:34.507690 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.508254 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.509083 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.509278 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.509353 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.510760 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:03:34.510763 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:03:34.510765 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:03:34.510813 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:03:34.512205 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.512592 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.512594 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.512595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.512664 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:03:34.512884 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:03:34.512886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:03:34.512887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:03:34.513060 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:03:34.513159 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:34.513341 (+ 182us) server_negotiation.cc:299] Negotiation successful
0923 18:03:34.513429 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:03:34.515900 31878 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:03:34.519204 31878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:03:34.526661 32000 tablet_service.cc:1467] Processing CreateTablet for tablet a7be9efefd744c4c80ef2d459ec80f5f (DEFAULT_TABLE table=test-table [id=95adb48846f34f89b992368206d025fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:03:34.526705 32139 tablet_service.cc:1467] Processing CreateTablet for tablet a7be9efefd744c4c80ef2d459ec80f5f (DEFAULT_TABLE table=test-table [id=95adb48846f34f89b992368206d025fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:03:34.527278 32000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7be9efefd744c4c80ef2d459ec80f5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:03:34.527330 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7be9efefd744c4c80ef2d459ec80f5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:03:34.530055 32097 tablet_bootstrap.cc:492] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194: Bootstrap starting.
I20260923 18:03:34.530069 31958 tablet_bootstrap.cc:492] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0: Bootstrap starting.
I20260923 18:03:34.530968 31958 tablet_bootstrap.cc:654] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0: Neither blocks nor log segments found. Creating new log.
I20260923 18:03:34.530977 32097 tablet_bootstrap.cc:654] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194: Neither blocks nor log segments found. Creating new log.
I20260923 18:03:34.531002 32073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.525627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:41273 (local address 127.0.0.1:55536)
0923 18:03:34.525787 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:34.525802 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:34.525880 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:34.526841 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:34.526844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:34.526863 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:03:34.526963 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.526969 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.528339 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.528342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.529547 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.529558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.530574 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.530578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.530665 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.530708 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 18:03:34.530793 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 18:03:34.530879 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260923 18:03:34.531046 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.525797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55536 (local address 127.15.139.131:41273)
0923 18:03:34.526220 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:34.526230 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:34.526269 (+ 39us) server_negotiation.cc:407] Connection header received
0923 18:03:34.526428 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:34.526434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:34.526656 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:34.526909 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:03:34.527099 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.528217 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.529683 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.530442 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.530576 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.530713 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:34.530823 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 18:03:34.530913 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":204,"threads_started":1}
I20260923 18:03:34.532701 31958 tablet_bootstrap.cc:492] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0: No bootstrap required, opened a new log
I20260923 18:03:34.532807 31958 ts_tablet_manager.cc:1397] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:03:34.532994 32097 tablet_bootstrap.cc:492] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194: No bootstrap required, opened a new log
I20260923 18:03:34.533100 32097 ts_tablet_manager.cc:1397] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 18:03:34.533288 31958 raft_consensus.cc:348] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } }
I20260923 18:03:34.533406 31958 raft_consensus.cc:374] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:03:34.533453 31958 raft_consensus.cc:729] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea49604a144312af4c718e7f4793b0, State: Initialized, Role: FOLLOWER
I20260923 18:03:34.533514 32097 raft_consensus.cc:348] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } }
I20260923 18:03:34.533625 32097 raft_consensus.cc:374] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:03:34.533579 31958 consensus_queue.cc:260] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } }
I20260923 18:03:34.533667 32097 raft_consensus.cc:729] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e945f5ab43b3400aaa03cf626edaf194, State: Initialized, Role: FOLLOWER
I20260923 18:03:34.533784 32097 consensus_queue.cc:260] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } }
I20260923 18:03:34.533849 31958 ts_tablet_manager.cc:1428] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260923 18:03:34.534448 32097 ts_tablet_manager.cc:1428] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 18:03:34.535424 32275 tablet_service.cc:1467] Processing CreateTablet for tablet a7be9efefd744c4c80ef2d459ec80f5f (DEFAULT_TABLE table=test-table [id=95adb48846f34f89b992368206d025fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:03:34.536638 32275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7be9efefd744c4c80ef2d459ec80f5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:03:34.543478 32353 tablet_bootstrap.cc:492] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0: Bootstrap starting.
I20260923 18:03:34.545976 32353 tablet_bootstrap.cc:654] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0: Neither blocks nor log segments found. Creating new log.
I20260923 18:03:34.546782 32353 log.cc:824] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0: Log is configured to *not* fsync() on all Append() calls
I20260923 18:03:34.548725 32353 tablet_bootstrap.cc:492] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0: No bootstrap required, opened a new log
I20260923 18:03:34.548933 32353 ts_tablet_manager.cc:1397] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260923 18:03:34.552091 32353 raft_consensus.cc:348] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } }
I20260923 18:03:34.552325 32353 raft_consensus.cc:374] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:03:34.552405 32353 raft_consensus.cc:729] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ad0f661238425f8932049c81fc48b0, State: Initialized, Role: FOLLOWER
I20260923 18:03:34.552838 32353 consensus_queue.cc:260] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } }
I20260923 18:03:34.553475 32341 heartbeater.cc:503] Master 127.15.139.190:36179 was elected leader, sending a full tablet report...
I20260923 18:03:34.553828 32353 ts_tablet_manager.cc:1428] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
W20260923 18:03:34.571656 32067 tablet.cc:2369] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:03:34.616726 32355 raft_consensus.cc:482] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:03:34.616950 32355 raft_consensus.cc:504] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } }
I20260923 18:03:34.618108 32355 leader_election.cc:288] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e945f5ab43b3400aaa03cf626edaf194 (127.15.139.130:45995), fdea49604a144312af4c718e7f4793b0 (127.15.139.129:35429)
I20260923 18:03:34.622465 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.618441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:35429 (local address 127.15.139.131:53437)
0923 18:03:34.618650 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:34.618668 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:34.618795 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:34.619096 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:34.619099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:34.619117 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:03:34.619294 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.619306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.620425 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.620456 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.621320 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.621328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.622050 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.622054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.622153 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.622175 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 18:03:34.622237 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 18:03:34.622307 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":6}
I20260923 18:03:34.622725 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.618553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:53437 (local address 127.15.139.129:35429)
0923 18:03:34.618809 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:34.618814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:34.618831 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:03:34.618890 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:34.618894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:34.618967 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:34.619086 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:03:34.619443 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.620315 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.621446 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.621915 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.622060 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.622224 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:34.622264 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 18:03:34.622329 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":84,"threads_started":1}
I20260923 18:03:34.622769 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.618684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:39717 (local address 127.15.139.130:45995)
0923 18:03:34.618900 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:34.618905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:34.619063 (+ 158us) server_negotiation.cc:407] Connection header received
0923 18:03:34.619247 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:34.619249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:34.619313 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:34.619425 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:03:34.619718 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.620525 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.621682 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.622154 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.622314 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.622466 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:34.622516 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 18:03:34.622602 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260923 18:03:34.622973 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.618471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:45995 (local address 127.15.139.131:39717)
0923 18:03:34.618962 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:34.618975 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:34.619117 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:34.619451 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:03:34.619453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:03:34.619470 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:03:34.619593 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.619599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.620673 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.620679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.621543 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:03:34.621553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.622297 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.622301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:03:34.622401 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.622427 (+ 26us) client_negotiation.cc:769] Sending connection context
0923 18:03:34.622502 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 18:03:34.622815 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":20,"thread_start_us":141,"threads_started":1}
I20260923 18:03:34.623453 32020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7be9efefd744c4c80ef2d459ec80f5f" candidate_uuid: "53ad0f661238425f8932049c81fc48b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea49604a144312af4c718e7f4793b0" is_pre_election: true
I20260923 18:03:34.623677 32160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7be9efefd744c4c80ef2d459ec80f5f" candidate_uuid: "53ad0f661238425f8932049c81fc48b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e945f5ab43b3400aaa03cf626edaf194" is_pre_election: true
I20260923 18:03:34.623852 32020 raft_consensus.cc:2454] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ad0f661238425f8932049c81fc48b0 in term 0.
I20260923 18:03:34.623960 32160 raft_consensus.cc:2454] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ad0f661238425f8932049c81fc48b0 in term 0.
I20260923 18:03:34.624431 32228 leader_election.cc:302] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 53ad0f661238425f8932049c81fc48b0, e945f5ab43b3400aaa03cf626edaf194, fdea49604a144312af4c718e7f4793b0; no voters:
I20260923 18:03:34.624735 32355 raft_consensus.cc:2781] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:03:34.624855 32355 raft_consensus.cc:482] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:03:34.624929 32355 raft_consensus.cc:3037] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:03:34.626102 32355 raft_consensus.cc:504] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } }
I20260923 18:03:34.626478 32355 leader_election.cc:288] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [CANDIDATE]: Term 1 election: Requested vote from peers e945f5ab43b3400aaa03cf626edaf194 (127.15.139.130:45995), fdea49604a144312af4c718e7f4793b0 (127.15.139.129:35429)
I20260923 18:03:34.626832 32020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7be9efefd744c4c80ef2d459ec80f5f" candidate_uuid: "53ad0f661238425f8932049c81fc48b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea49604a144312af4c718e7f4793b0"
I20260923 18:03:34.626828 32160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7be9efefd744c4c80ef2d459ec80f5f" candidate_uuid: "53ad0f661238425f8932049c81fc48b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e945f5ab43b3400aaa03cf626edaf194"
I20260923 18:03:34.626963 32020 raft_consensus.cc:3037] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:03:34.626965 32160 raft_consensus.cc:3037] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:03:34.628242 32020 raft_consensus.cc:2454] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ad0f661238425f8932049c81fc48b0 in term 1.
I20260923 18:03:34.628242 32160 raft_consensus.cc:2454] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ad0f661238425f8932049c81fc48b0 in term 1.
I20260923 18:03:34.628551 32231 leader_election.cc:302] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ad0f661238425f8932049c81fc48b0, fdea49604a144312af4c718e7f4793b0; no voters:
I20260923 18:03:34.628726 32355 raft_consensus.cc:2781] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:03:34.628950 32355 raft_consensus.cc:686] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 1 LEADER]: Becoming Leader. State: Replica: 53ad0f661238425f8932049c81fc48b0, State: Running, Role: LEADER
I20260923 18:03:34.629279 32355 consensus_queue.cc:237] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } }
I20260923 18:03:34.632400 31874 catalog_manager.cc:5495] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ad0f661238425f8932049c81fc48b0 (127.15.139.131). New cstate: current_term: 1 leader_uuid: "53ad0f661238425f8932049c81fc48b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ad0f661238425f8932049c81fc48b0" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 41273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 } health_report { overall_health: UNKNOWN } } }
I20260923 18:03:34.656368 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:34.652910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55542 (local address 127.15.139.131:41273)
0923 18:03:34.653058 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0923 18:03:34.653061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:03:34.653075 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:03:34.653210 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:03:34.653212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:03:34.653266 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:03:34.653359 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:03:34.653600 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.654112 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.654963 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:03:34.655169 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:03:34.655245 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:03:34.655586 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:03:34.656026 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:03:34.656120 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 18:03:34.656193 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 18:03:34.656241 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 18:03:34.662259 32020 raft_consensus.cc:1261] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0 [term 1 FOLLOWER]: Refusing update from remote peer 53ad0f661238425f8932049c81fc48b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:03:34.662272 32160 raft_consensus.cc:1261] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194 [term 1 FOLLOWER]: Refusing update from remote peer 53ad0f661238425f8932049c81fc48b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:03:34.663007 32355 consensus_queue.cc:1035] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea49604a144312af4c718e7f4793b0" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 35429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:03:34.663205 32358 consensus_queue.cc:1035] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e945f5ab43b3400aaa03cf626edaf194" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:03:34.668282 32361 mvcc.cc:205] Tried to move back new op lower bound from 7332604373649526784 to 7332604373524254720. Current Snapshot: MvccSnapshot[applied={T|T < 7332604373649526784}]
I20260923 18:03:34.668275 32362 mvcc.cc:205] Tried to move back new op lower bound from 7332604373649526784 to 7332604373524254720. Current Snapshot: MvccSnapshot[applied={T|T < 7332604373649526784}]
I20260923 18:03:34.676899 31878 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:03:34.677050 31878 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:03:34.679593 31878 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Sending DeleteTablet for 3 replicas of tablet a7be9efefd744c4c80ef2d459ec80f5f
I20260923 18:03:34.680243 32139 tablet_service.cc:1514] Processing DeleteTablet for tablet a7be9efefd744c4c80ef2d459ec80f5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:03:34 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:51706
I20260923 18:03:34.680251 32275 tablet_service.cc:1514] Processing DeleteTablet for tablet a7be9efefd744c4c80ef2d459ec80f5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:03:34 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:55536
I20260923 18:03:34.680439 32000 tablet_service.cc:1514] Processing DeleteTablet for tablet a7be9efefd744c4c80ef2d459ec80f5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:03:34 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:55958
I20260923 18:03:34.680493 32212 tablet_replica.cc:330] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194: stopping tablet replica
I20260923 18:03:34.680620 32212 raft_consensus.cc:2229] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:03:34.680760 32212 raft_consensus.cc:2258] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:03:34.680780 32372 tablet_replica.cc:330] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0: stopping tablet replica
I20260923 18:03:34.680903 32372 raft_consensus.cc:2229] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:03:34.681016 32371 tablet_replica.cc:330] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0: stopping tablet replica
I20260923 18:03:34.681041 32372 raft_consensus.cc:2258] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:03:34.681356 32371 raft_consensus.cc:2229] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:03:34.681721 32212 ts_tablet_manager.cc:1905] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:03:34.681792 32372 ts_tablet_manager.cc:1905] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:03:34.681921 32371 raft_consensus.cc:2258] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:03:34.683599 32371 ts_tablet_manager.cc:1905] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:03:34.684196 32212 ts_tablet_manager.cc:1918] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:03:34.684196 32372 ts_tablet_manager.cc:1918] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:03:34.684279 32212 log.cc:1196] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/a7be9efefd744c4c80ef2d459ec80f5f
I20260923 18:03:34.684279 32372 log.cc:1196] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal/wals/a7be9efefd744c4c80ef2d459ec80f5f
I20260923 18:03:34.684554 32212 ts_tablet_manager.cc:1939] T a7be9efefd744c4c80ef2d459ec80f5f P e945f5ab43b3400aaa03cf626edaf194: Deleting consensus metadata
I20260923 18:03:34.684554 32372 ts_tablet_manager.cc:1939] T a7be9efefd744c4c80ef2d459ec80f5f P fdea49604a144312af4c718e7f4793b0: Deleting consensus metadata
I20260923 18:03:34.685204 31865 catalog_manager.cc:4841] TS fdea49604a144312af4c718e7f4793b0 (127.15.139.129:35429): tablet a7be9efefd744c4c80ef2d459ec80f5f (table test-table [id=95adb48846f34f89b992368206d025fa]) successfully deleted
I20260923 18:03:34.685397 31862 catalog_manager.cc:4841] TS e945f5ab43b3400aaa03cf626edaf194 (127.15.139.130:45995): tablet a7be9efefd744c4c80ef2d459ec80f5f (table test-table [id=95adb48846f34f89b992368206d025fa]) successfully deleted
I20260923 18:03:34.685474 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 31916
I20260923 18:03:34.686550 32371 ts_tablet_manager.cc:1918] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:03:34.686642 32371 log.cc:1196] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/a7be9efefd744c4c80ef2d459ec80f5f
I20260923 18:03:34.686986 32371 ts_tablet_manager.cc:1939] T a7be9efefd744c4c80ef2d459ec80f5f P 53ad0f661238425f8932049c81fc48b0: Deleting consensus metadata
I20260923 18:03:34.688246 31863 catalog_manager.cc:4841] TS 53ad0f661238425f8932049c81fc48b0 (127.15.139.131:41273): tablet a7be9efefd744c4c80ef2d459ec80f5f (table test-table [id=95adb48846f34f89b992368206d025fa]) successfully deleted
I20260923 18:03:34.693037 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 32071
I20260923 18:03:34.699975 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 32209
I20260923 18:03:34.705865 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 31845
I20260923 18:03:54.712754 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36179
--webserver_interface=127.15.139.190
--webserver_port=33647
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:36179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:03:54.831453 32373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:03:54.831760 32373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:03:54.831804 32373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:03:54.836046 32373 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 18:03:54.836118 32373 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 18:03:54.836135 32373 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 18:03:54.836166 32373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:03:54.836221 32373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:03:54.836266 32373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:03:54.836295 32373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:03:54.836333 32373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:03:54.841946 32373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:36179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=33647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:03:54.843063 32373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:03:54.844189 32373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 18:03:54.853520 32373 server_base.cc:1034] running on GCE node
W20260923 18:03:54.853576 32381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:54.853543 32379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:54.853543 32378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:54.854419 32373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:03:54.855782 32373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:03:54.856988 32373 hybrid_clock.cc:648] HybridClock initialized: now 1790186634856950 us; error 52 us; skew 500 ppm
Sep 23 18:03:54 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:03:54.865684 32373 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:03:54.866689 32373 webserver.cc:460] Webserver started at http://127.15.139.190:33647/ using document root <none> and password file <none>
I20260923 18:03:54.867300 32373 fs_manager.cc:359] Metadata directory not provided
I20260923 18:03:54.867398 32373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:03:54.871588 32373 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260923 18:03:54.873843 32388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:54.875152 32373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260923 18:03:54.875296 32373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "01dbfc16e7f74aa0b6047b3db1e030ec"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "5f01429844ed5a080ae428c8cf8bc0a4"
server_key_iv: "897cb93b1b0dfbc91b092064f3ff0faa"
server_key_version: "encryptionkey@0"
I20260923 18:03:54.876052 32373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:03:54.887321 32373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:03:54.890496 32373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:03:54.890715 32373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:03:54.897749 32373 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:36179
I20260923 18:03:54.898026 32440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:36179 every 8 connection(s)
I20260923 18:03:54.898836 32373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:03:54.900131 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 32373
I20260923 18:03:54.900599 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:35429
--local_ip_for_outbound_sockets=127.15.139.129
--tserver_master_addrs=127.15.139.190:36179
--webserver_port=43697
--webserver_interface=127.15.139.129
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:03:54.903865 32441 sys_catalog.cc:263] Verifying existing consensus state
I20260923 18:03:54.905620 32441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Bootstrap starting.
I20260923 18:03:54.923188 32441 log.cc:824] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Log is configured to *not* fsync() on all Append() calls
I20260923 18:03:54.934602 32441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 18:03:54.935081 32441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Bootstrap complete.
I20260923 18:03:54.939304 32441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } }
I20260923 18:03:54.939790 32441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01dbfc16e7f74aa0b6047b3db1e030ec, State: Initialized, Role: FOLLOWER
I20260923 18:03:54.940260 32441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } }
I20260923 18:03:54.940412 32441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:03:54.940500 32441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:03:54.940611 32441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 2 FOLLOWER]: Advancing to term 3
I20260923 18:03:54.942513 32441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } }
I20260923 18:03:54.942852 32441 leader_election.cc:302] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01dbfc16e7f74aa0b6047b3db1e030ec; no voters:
I20260923 18:03:54.943197 32441 leader_election.cc:288] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 18:03:54.943305 32445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 3 FOLLOWER]: Leader election won for term 3
I20260923 18:03:54.943547 32445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [term 3 LEADER]: Becoming Leader. State: Replica: 01dbfc16e7f74aa0b6047b3db1e030ec, State: Running, Role: LEADER
I20260923 18:03:54.943863 32445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } }
I20260923 18:03:54.944506 32441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:03:54.944857 32445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01dbfc16e7f74aa0b6047b3db1e030ec. Latest consensus state: current_term: 3 leader_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } } }
I20260923 18:03:54.944984 32445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: This master's current role is: LEADER
I20260923 18:03:54.945454 32446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01dbfc16e7f74aa0b6047b3db1e030ec" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36179 } } }
I20260923 18:03:54.945627 32446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec [sys.catalog]: This master's current role is: LEADER
I20260923 18:03:54.946354 32449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:03:54.951488 32449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:03:54.952188 32449 catalog_manager.cc:1259] Loaded cluster ID: beb790a1cbb4499aa0542671d5fbde91
I20260923 18:03:54.952250 32449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:03:54.954792 32449 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:03:54.955600 32449 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Loaded TSK: 0
I20260923 18:03:54.955685 32449 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 01dbfc16e7f74aa0b6047b3db1e030ec: Loaded TSK: 1
F20260923 18:03:54.956457 32449 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f96bb41f9ad google::LogMessage::Fail() at ??:0
@ 0x7f96bb42288a google::LogMessage::SendToLog() at ??:0
@ 0x7f96bb41f4de google::LogMessage::Flush() at ??:0
@ 0x7f96bb420a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f96be0e318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f96be0dbf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f96c5d887bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f96c5d6299c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f96c5d63178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f96c5da4df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f96c5dc382b std::function<>::operator()() at ??:0
@ 0x7f96c5d62b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f96c5d63f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f96c5d60579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f96c5da4640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f96bd6713d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f96bd671c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f96bd6733f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f96bd662512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f96beca66db start_thread at ??:0
@ 0x7f96b9f1d61f clone at ??:0
W20260923 18:03:55.057561 32443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:03:55.057942 32443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:03:55.058058 32443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:03:55.062740 32443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:03:55.062803 32443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:03:55.062906 32443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:03:55.068388 32443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:35429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=43697
--tserver_master_addrs=127.15.139.190:36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:03:55.069446 32443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:03:55.070475 32443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:03:55.080196 32488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:55.080228 32489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:55.080474 32443 server_base.cc:1034] running on GCE node
W20260923 18:03:55.080560 32491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:55.081035 32443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:03:55.081763 32443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:03:55.083047 32443 hybrid_clock.cc:648] HybridClock initialized: now 1790186635083031 us; error 42 us; skew 500 ppm
Sep 23 18:03:55 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186635, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:03:55.085924 32443 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:03:55.086889 32443 webserver.cc:460] Webserver started at http://127.15.139.129:43697/ using document root <none> and password file <none>
I20260923 18:03:55.087461 32443 fs_manager.cc:359] Metadata directory not provided
I20260923 18:03:55.087589 32443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:03:55.091559 32443 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260923 18:03:55.093688 32498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:55.094990 32443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:03:55.095080 32443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "fdea49604a144312af4c718e7f4793b0"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "68bca8fee8b924f7e36a751972154feb"
server_key_iv: "a8da278e9a55da42f4c9466a97fc832e"
server_key_version: "encryptionkey@0"
I20260923 18:03:55.095688 32443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:03:55.114845 32443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:03:55.118494 32443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:03:55.118742 32443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:03:55.119555 32443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:03:55.120612 32443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:03:55.120674 32443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:55.120738 32443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:03:55.120771 32443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:55.133420 32443 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:35429
I20260923 18:03:55.133441 32611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:35429 every 8 connection(s)
I20260923 18:03:55.134588 32443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:03:55.138473 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 32443
I20260923 18:03:55.138929 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:45995
--local_ip_for_outbound_sockets=127.15.139.130
--tserver_master_addrs=127.15.139.190:36179
--webserver_port=37607
--webserver_interface=127.15.139.130
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:03:55.191121 32614 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:03:55.137657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:47389)
0923 18:03:55.138509 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:55.138632 (+ 123us) client_negotiation.cc:174] Beginning negotiation
0923 18:03:55.139804 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:03:55.190156 (+ 50352us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":117,"threads_started":1}
W20260923 18:03:55.191771 32612 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:36179 (0 consecutive failures): Network error: Failed to ping master at 127.15.139.190:36179: Client connection negotiation failed: client connection to 127.15.139.190:36179: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 18:03:55.192544 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:55.192250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:32897)
0923 18:03:55.192400 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:55.192479 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260923 18:03:55.193271 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:55.193045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:60377)
0923 18:03:55.193154 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:55.193209 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 18:03:55.193472 32612 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 18:03:55.269896 32615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:03:55.270156 32615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:03:55.270232 32615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:03:55.274356 32615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:03:55.274423 32615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:03:55.274533 32615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:03:55.279768 32615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:45995
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=37607
--tserver_master_addrs=127.15.139.190:36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:03:55.280768 32615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:03:55.281769 32615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:03:55.291604 32620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:55.291673 32615 server_base.cc:1034] running on GCE node
W20260923 18:03:55.291604 32621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:55.291644 32623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:55.292346 32615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:03:55.293113 32615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:03:55.294308 32615 hybrid_clock.cc:648] HybridClock initialized: now 1790186635294291 us; error 26 us; skew 500 ppm
Sep 23 18:03:55 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186635, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:03:55.297281 32615 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:03:55.298273 32615 webserver.cc:460] Webserver started at http://127.15.139.130:37607/ using document root <none> and password file <none>
I20260923 18:03:55.298863 32615 fs_manager.cc:359] Metadata directory not provided
I20260923 18:03:55.298929 32615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:03:55.302865 32615 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:03:55.305006 32630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:55.306305 32615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:03:55.306509 32615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "e945f5ab43b3400aaa03cf626edaf194"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "a5d8ccd49eae7b3e629371518d29ad27"
server_key_iv: "2b649ffe75ea57bddf2cffd9947e9b65"
server_key_version: "encryptionkey@0"
I20260923 18:03:55.307106 32615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:03:55.319280 32615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:03:55.322232 32615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:03:55.322439 32615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:03:55.323175 32615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:03:55.324344 32615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:03:55.324411 32615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:55.324473 32615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:03:55.324505 32615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:55.333997 32615 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:45995
I20260923 18:03:55.334028 32743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:45995 every 8 connection(s)
I20260923 18:03:55.335134 32615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:03:55.336558 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 32615
I20260923 18:03:55.337070 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:41273
--local_ip_for_outbound_sockets=127.15.139.131
--tserver_master_addrs=127.15.139.190:36179
--webserver_port=35707
--webserver_interface=127.15.139.131
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:03:55.339669 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:55.337615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:54483)
0923 18:03:55.338523 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:55.338709 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":163,"threads_started":1}
W20260923 18:03:55.340189 32744 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:36179 (0 consecutive failures): Network error: Failed to ping master at 127.15.139.190:36179: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
I20260923 18:03:55.341001 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:55.340673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:50143)
0923 18:03:55.340828 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:55.340905 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260923 18:03:55.341698 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:55.341488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:44715)
0923 18:03:55.341593 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:55.341640 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 18:03:55.342062 32744 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 18:03:55.462404 32747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:03:55.462670 32747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:03:55.462713 32747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:03:55.466809 32747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:03:55.466874 32747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:03:55.466956 32747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:03:55.472227 32747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:41273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=35707
--tserver_master_addrs=127.15.139.190:36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:03:55.473284 32747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:03:55.474386 32747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:03:55.483973 32752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:55.483973 32753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:03:55.484073 32755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:03:55.484342 32747 server_base.cc:1034] running on GCE node
I20260923 18:03:55.484803 32747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:03:55.485451 32747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:03:55.486644 32747 hybrid_clock.cc:648] HybridClock initialized: now 1790186635486627 us; error 32 us; skew 500 ppm
Sep 23 18:03:55 dist-test-slave-vr57 krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186635, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:03:55.489732 32747 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:03:55.490654 32747 webserver.cc:460] Webserver started at http://127.15.139.131:35707/ using document root <none> and password file <none>
I20260923 18:03:55.491225 32747 fs_manager.cc:359] Metadata directory not provided
I20260923 18:03:55.491314 32747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:03:55.495285 32747 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:03:55.497498 32762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:55.498746 32747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:03:55.498885 32747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "53ad0f661238425f8932049c81fc48b0"
format_stamp: "Formatted at 2026-09-23 18:03:12 on dist-test-slave-vr57"
server_key: "ad0815028e599ab4f2c150c4710c7ec1"
server_key_iv: "b34e4c9a6b69acb3145d5d56e9300ec1"
server_key_version: "encryptionkey@0"
I20260923 18:03:55.499631 32747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:03:55.521255 32747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:03:55.524684 32747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:03:55.524917 32747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:03:55.525573 32747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:03:55.526475 32747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:03:55.526549 32747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:55.526628 32747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:03:55.526671 32747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:03:55.536505 32747 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:41273
I20260923 18:03:55.536533 407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:41273 every 8 connection(s)
I20260923 18:03:55.537580 32747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:03:55.541810 410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:55.539866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:41703)
0923 18:03:55.540363 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:55.540482 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":119,"threads_started":1}
W20260923 18:03:55.542315 408 heartbeater.cc:644] Failed to heartbeat to 127.15.139.190:36179 (0 consecutive failures): Network error: Failed to ping master at 127.15.139.190:36179: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
I20260923 18:03:55.542932 410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:55.542697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:50767)
0923 18:03:55.542802 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:55.542852 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 18:03:55.543624 410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:55.543356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:34489)
0923 18:03:55.543476 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:55.543576 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260923 18:03:55.543862 408 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 18:03:55.545197 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 32747
I20260923 18:03:56.194813 412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:56.194210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:49851)
0923 18:03:56.194579 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:56.194681 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":83,"threads_started":1}
I20260923 18:03:56.342953 413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:56.342577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:56259)
0923 18:03:56.342795 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:56.342848 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":69,"threads_started":1}
I20260923 18:03:56.544811 414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:56.544391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:52523)
0923 18:03:56.544646 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:56.544705 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":70,"threads_started":1}
I20260923 18:03:57.196202 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:57.195726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:55197)
0923 18:03:57.196024 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:57.196083 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":84,"threads_started":1}
I20260923 18:03:57.344327 416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:57.343890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:42101)
0923 18:03:57.344153 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:57.344213 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":76,"threads_started":1}
I20260923 18:03:57.546106 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:57.545603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:52813)
0923 18:03:57.545877 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:57.545970 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":79,"threads_started":1}
I20260923 18:03:58.197525 418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:58.197102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:33365)
0923 18:03:58.197356 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:58.197416 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":77,"threads_started":1}
I20260923 18:03:58.345636 419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:58.345179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:60289)
0923 18:03:58.345459 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:58.345530 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":72,"threads_started":1}
I20260923 18:03:58.547369 420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:58.546898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:52461)
0923 18:03:58.547207 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:58.547272 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":78,"threads_started":1}
I20260923 18:03:59.198837 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:59.198352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:47079)
0923 18:03:59.198631 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:59.198709 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":74,"threads_started":1}
I20260923 18:03:59.346951 422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:59.346465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:32929)
0923 18:03:59.346771 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:59.346830 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":105,"threads_started":1}
I20260923 18:03:59.548674 423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:03:59.548197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:38171)
0923 18:03:59.548515 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0923 18:03:59.548578 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":98,"threads_started":1}
I20260923 18:04:00.200271 424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:00.199765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:49079)
0923 18:04:00.200078 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:00.200153 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":149,"threads_started":1}
I20260923 18:04:00.348389 425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:00.347947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:55915)
0923 18:04:00.348228 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:00.348290 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":88,"threads_started":1}
I20260923 18:04:00.549945 426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:00.549422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:54351)
0923 18:04:00.549763 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:00.549831 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":83,"threads_started":1}
I20260923 18:04:01.201579 427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:01.201056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:38159)
0923 18:04:01.201348 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:01.201475 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":86,"threads_started":1}
I20260923 18:04:01.349694 428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:01.349156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:46283)
0923 18:04:01.349472 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:01.349560 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":97,"threads_started":1}
I20260923 18:04:01.551244 429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:01.550744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:42459)
0923 18:04:01.551062 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:01.551135 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":78,"threads_started":1}
I20260923 18:04:02.202955 430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:02.202473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:44577)
0923 18:04:02.202794 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:02.202857 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":83,"threads_started":1}
I20260923 18:04:02.351230 431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:02.350580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:55661)
0923 18:04:02.350970 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:02.351067 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":106,"threads_started":1}
I20260923 18:04:02.552667 432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:02.552140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:36843)
0923 18:04:02.552489 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:02.552561 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":74,"threads_started":1}
I20260923 18:04:03.204376 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:03.203866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:47451)
0923 18:04:03.204178 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:03.204263 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":87,"threads_started":1}
I20260923 18:04:03.352571 434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:03.352127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:58597)
0923 18:04:03.352401 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:03.352469 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":88,"threads_started":1}
I20260923 18:04:03.553964 435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:03.553512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:40059)
0923 18:04:03.553795 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:03.553861 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":79,"threads_started":1}
I20260923 18:04:04.205479 436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:04.205074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:44401)
0923 18:04:04.205325 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:04.205381 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260923 18:04:04.353919 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:04.353465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:56243)
0923 18:04:04.353741 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:04.353803 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":105,"threads_started":1}
I20260923 18:04:04.555249 438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:04.554826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:48509)
0923 18:04:04.555101 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:04.555156 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":88,"threads_started":1}
I20260923 18:04:05.206708 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:05.206269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:37993)
0923 18:04:05.206522 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:05.206608 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
I20260923 18:04:05.355201 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:05.354778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:47189)
0923 18:04:05.355037 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:05.355101 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":79,"threads_started":1}
I20260923 18:04:05.556653 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:05.556123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:49505)
0923 18:04:05.556494 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:05.556556 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":162,"threads_started":1}
I20260923 18:04:06.207849 442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:06.207388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:39507)
0923 18:04:06.207697 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:06.207752 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":135,"threads_started":1}
I20260923 18:04:06.356822 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:06.356299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:53895)
0923 18:04:06.356658 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:06.356724 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":151,"threads_started":1}
I20260923 18:04:06.557971 444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:06.557570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:46841)
0923 18:04:06.557827 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:06.557879 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":85,"threads_started":1}
I20260923 18:04:07.209129 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:07.208674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:56235)
0923 18:04:07.208972 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:07.209032 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":88,"threads_started":1}
I20260923 18:04:07.358145 446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:07.357694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:57813)
0923 18:04:07.357976 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:07.358047 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":84,"threads_started":1}
I20260923 18:04:07.559391 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:07.558920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:40945)
0923 18:04:07.559232 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:07.559291 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":89,"threads_started":1}
I20260923 18:04:08.210526 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:08.209905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:52793)
0923 18:04:08.210214 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:08.210272 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":82,"threads_started":1}
I20260923 18:04:08.359452 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:08.358991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:60431)
0923 18:04:08.359285 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:08.359350 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":80,"threads_started":1}
I20260923 18:04:08.560797 450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:08.560319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:40537)
0923 18:04:08.560650 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:08.560705 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":97,"threads_started":1}
I20260923 18:04:09.211967 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:09.211407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:45957)
0923 18:04:09.211772 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:09.211860 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":158,"threads_started":1}
I20260923 18:04:09.360925 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:09.360424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:60379)
0923 18:04:09.360761 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:09.360825 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":95,"threads_started":1}
I20260923 18:04:09.562376 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:09.561849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.131:52771)
0923 18:04:09.562197 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:09.562269 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":114,"threads_started":1}
I20260923 18:04:10.213613 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:10.213137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.129:38115)
0923 18:04:10.213445 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:10.213518 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":93,"threads_started":1}
I20260923 18:04:10.362500 455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:10.361938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36179 (local address 127.15.139.130:39761)
0923 18:04:10.362270 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:10.362342 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.190:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":110,"threads_started":1}
I20260923 18:04:10.548830 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 32443
I20260923 18:04:10.555124 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 32615
I20260923 18:04:10.560668 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 32747
2026-09-23T18:04:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59731 ms)
[----------] 30 tests from SecurityITest (251688 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:10 dist-test-slave-vr57 krb5kdc[458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:10 dist-test-slave-vr57 krb5kdc[458](info): set up 2 sockets
Sep 23 18:04:10 dist-test-slave-vr57 krb5kdc[458](info): commencing operation
krb5kdc: starting...
W20260923 18:04:11.652529 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:11 dist-test-slave-vr57 krb5kdc[458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:04:11.782789 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:46077
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:46077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:11.895185 471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:11.895455 471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:11.895543 471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:11.899616 471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:11.899672 471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:11.899693 471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:11.899708 471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:11.899760 471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:11.905046 471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:46077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:46077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:11.906140 471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:11.907249 471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 18:04:11.916514 471 server_base.cc:1034] running on GCE node
W20260923 18:04:11.916569 479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:11.916568 477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:11.916597 476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:11.917450 471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:11.918624 471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:11.919799 471 hybrid_clock.cc:648] HybridClock initialized: now 1790186651919751 us; error 60 us; skew 500 ppm
Sep 23 18:04:11 dist-test-slave-vr57 krb5kdc[458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186651, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:11.922818 471 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:04:11.923853 471 webserver.cc:460] Webserver started at http://127.15.139.190:38231/ using document root <none> and password file <none>
I20260923 18:04:11.924428 471 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:11.924491 471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:11.924746 471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:11.926822 471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "58b1fce3cbe74e8898a3f27fcbf971a3"
format_stamp: "Formatted at 2026-09-23 18:04:11 on dist-test-slave-vr57"
server_key: "2c0e44dca310cb5ad23f054ffb248460"
server_key_iv: "9e3b0cf661b1a11cfa411e21d30de76a"
server_key_version: "encryptionkey@0"
I20260923 18:04:11.927435 471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "58b1fce3cbe74e8898a3f27fcbf971a3"
format_stamp: "Formatted at 2026-09-23 18:04:11 on dist-test-slave-vr57"
server_key: "2c0e44dca310cb5ad23f054ffb248460"
server_key_iv: "9e3b0cf661b1a11cfa411e21d30de76a"
server_key_version: "encryptionkey@0"
I20260923 18:04:11.931236 471 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:11.933842 486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:11.935105 471 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260923 18:04:11.935263 471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "58b1fce3cbe74e8898a3f27fcbf971a3"
format_stamp: "Formatted at 2026-09-23 18:04:11 on dist-test-slave-vr57"
server_key: "2c0e44dca310cb5ad23f054ffb248460"
server_key_iv: "9e3b0cf661b1a11cfa411e21d30de76a"
server_key_version: "encryptionkey@0"
I20260923 18:04:11.935384 471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:11.948094 471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:11.951314 471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:11.951592 471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:11.959476 471 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:46077
I20260923 18:04:11.959684 538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:46077 every 8 connection(s)
I20260923 18:04:11.960639 471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:11.963603 539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:11.969450 539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3: Bootstrap starting.
I20260923 18:04:11.969935 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 471
I20260923 18:04:11.970095 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:11.970372 15918 external_mini_cluster.cc:1428] Setting key 06246ef6893ae170f8152f65d10eae4a
I20260923 18:04:11.972411 539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:11.973387 539 log.cc:824] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:11.975965 539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3: No bootstrap required, opened a new log
I20260923 18:04:11.980427 539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58b1fce3cbe74e8898a3f27fcbf971a3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 46077 } }
I20260923 18:04:11.980717 539 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:11.980819 539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58b1fce3cbe74e8898a3f27fcbf971a3, State: Initialized, Role: FOLLOWER
Sep 23 18:04:11 dist-test-slave-vr57 krb5kdc[458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:11.981397 539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58b1fce3cbe74e8898a3f27fcbf971a3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 46077 } }
I20260923 18:04:11.981603 539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:11.981684 539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:11.981822 539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:11.983268 539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58b1fce3cbe74e8898a3f27fcbf971a3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 46077 } }
I20260923 18:04:11.983747 539 leader_election.cc:302] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58b1fce3cbe74e8898a3f27fcbf971a3; no voters:
I20260923 18:04:11.984197 539 leader_election.cc:288] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:11.984330 544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:11.984582 544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [term 1 LEADER]: Becoming Leader. State: Replica: 58b1fce3cbe74e8898a3f27fcbf971a3, State: Running, Role: LEADER
I20260923 18:04:11.985008 544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58b1fce3cbe74e8898a3f27fcbf971a3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 46077 } }
I20260923 18:04:11.985770 539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:11.986218 544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58b1fce3cbe74e8898a3f27fcbf971a3. Latest consensus state: current_term: 1 leader_uuid: "58b1fce3cbe74e8898a3f27fcbf971a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58b1fce3cbe74e8898a3f27fcbf971a3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 46077 } } }
I20260923 18:04:11.986441 544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:11.986702 546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58b1fce3cbe74e8898a3f27fcbf971a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58b1fce3cbe74e8898a3f27fcbf971a3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 46077 } } }
I20260923 18:04:11.986799 546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:11.986987 549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:11.989698 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:11.971895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33232 (local address 127.15.139.190:46077)
0923 18:04:11.972406 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:11.972430 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:11.972481 (+ 51us) server_negotiation.cc:407] Connection header received
0923 18:04:11.973455 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:11.973482 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:11.973946 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:11.974274 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:11.975116 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:11.976638 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:11.977761 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:11.978526 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:11.978716 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:11.982254 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:11.982261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:11.982282 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:11.982329 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:11.984791 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:11.985495 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:11.985500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:11.985506 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:11.985664 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:11.986080 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:11.986083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:11.986085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:11.986708 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:11.987895 (+ 1187us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:11.988103 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 18:04:11.988592 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":120,"threads_started":1}
I20260923 18:04:11.990655 549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:11.996490 549 catalog_manager.cc:1347] Generated new cluster ID: bf73c7effd9f4acf8276c9d13f4903ef
I20260923 18:04:11.996577 549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:12.021150 549 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:12.022557 549 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:12.036554 549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3: Generated new TSK 0
I20260923 18:04:12.037528 549 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:04:12.067641 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:46077
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:12.183635 565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:12.183916 565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:12.183961 565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:12.188565 565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:12.188654 565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:12.188716 565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:04:12.194103 565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:12.195150 565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:12.196240 565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:12.206120 571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:12.206120 570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:12.206146 573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:12.206413 565 server_base.cc:1034] running on GCE node
I20260923 18:04:12.206951 565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:12.207813 565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:12.209008 565 hybrid_clock.cc:648] HybridClock initialized: now 1790186652208981 us; error 33 us; skew 500 ppm
Sep 23 18:04:12 dist-test-slave-vr57 krb5kdc[458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186652, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:12.211974 565 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:04:12.212880 565 webserver.cc:460] Webserver started at http://127.15.139.129:39723/ using document root <none> and password file <none>
I20260923 18:04:12.213474 565 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:12.213587 565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:12.213832 565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:12.215979 565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "ba97fed20a42408281bdbdd6e0188d98"
format_stamp: "Formatted at 2026-09-23 18:04:12 on dist-test-slave-vr57"
server_key: "16b2816d1a6b2c974f78ee6b1390758d"
server_key_iv: "b6d0b70e1a65d3354ac4440943d60e7a"
server_key_version: "encryptionkey@0"
I20260923 18:04:12.216564 565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "ba97fed20a42408281bdbdd6e0188d98"
format_stamp: "Formatted at 2026-09-23 18:04:12 on dist-test-slave-vr57"
server_key: "16b2816d1a6b2c974f78ee6b1390758d"
server_key_iv: "b6d0b70e1a65d3354ac4440943d60e7a"
server_key_version: "encryptionkey@0"
I20260923 18:04:12.220417 565 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:04:12.223292 580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:12.224668 565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:12.224821 565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "ba97fed20a42408281bdbdd6e0188d98"
format_stamp: "Formatted at 2026-09-23 18:04:12 on dist-test-slave-vr57"
server_key: "16b2816d1a6b2c974f78ee6b1390758d"
server_key_iv: "b6d0b70e1a65d3354ac4440943d60e7a"
server_key_version: "encryptionkey@0"
I20260923 18:04:12.224941 565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:12.256832 565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:12.260269 565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:12.260516 565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:12.261191 565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:12.262120 565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:12.262202 565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:12.262313 565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:12.262370 565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:12.272143 565 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:42025
I20260923 18:04:12.272164 693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:42025 every 8 connection(s)
I20260923 18:04:12.273164 565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:04:12.275333 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 565
I20260923 18:04:12.275471 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:04:12.275831 15918 external_mini_cluster.cc:1428] Setting key 3c98ab47304106bd6552c44139ba5fa7
Sep 23 18:04:12 dist-test-slave-vr57 krb5kdc[458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186652, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:04:12.289455 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.275044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:60777 (local address 127.15.139.190:46077)
0923 18:04:12.275205 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:12.275209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:12.276384 (+ 1175us) server_negotiation.cc:407] Connection header received
0923 18:04:12.278171 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:12.278175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:12.278228 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:12.278332 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:12.279928 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.280459 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.281719 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.281985 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.282106 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.285125 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:12.285129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:12.285131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:12.285166 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:12.287113 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:12.287695 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:12.287698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:12.287699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:12.287775 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:12.288189 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:12.288191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:12.288192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:12.288372 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:12.288448 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:12.289009 (+ 561us) server_negotiation.cc:299] Negotiation successful
0923 18:04:12.289187 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 18:04:12.290452 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.275617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:46077 (local address 127.15.139.129:60777)
0923 18:04:12.276173 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:12.276244 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:12.277941 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:12.278517 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:12.278524 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:12.278954 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:12.279735 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:12.279758 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.280590 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.280593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:12.281549 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:12.281560 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.282152 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.282156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:12.282373 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.283095 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:12.283121 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:12.284932 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:12.287230 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:12.287235 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:12.287249 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:12.287557 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:12.287839 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:12.287841 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:12.287843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:12.288055 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:12.288461 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:12.288467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:12.288755 (+ 288us) client_negotiation.cc:769] Sending connection context
0923 18:04:12.288918 (+ 163us) client_negotiation.cc:240] Negotiation successful
0923 18:04:12.289316 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":150,"threads_started":1}
I20260923 18:04:12.291926 694 heartbeater.cc:347] Connected to a master server at 127.15.139.190:46077
I20260923 18:04:12.292235 694 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:12.292876 694 heartbeater.cc:511] Master 127.15.139.190:46077 requested a full tablet report, sending...
I20260923 18:04:12.294540 503 ts_manager.cc:194] Registered new tserver with Master: ba97fed20a42408281bdbdd6e0188d98 (127.15.139.129:42025)
I20260923 18:04:12.295845 503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:60777
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:04:12.306555 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:46077
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:12.421185 699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:12.421516 699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:12.421593 699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:12.425953 699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:12.426079 699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:12.426182 699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:04:12.431965 699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:12.433101 699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:12.434219 699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:12.443997 704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:12.444002 707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:12.444144 705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:12.444540 699 server_base.cc:1034] running on GCE node
I20260923 18:04:12.444977 699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:12.445722 699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:12.446926 699 hybrid_clock.cc:648] HybridClock initialized: now 1790186652446904 us; error 35 us; skew 500 ppm
Sep 23 18:04:12 dist-test-slave-vr57 krb5kdc[458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186652, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:12.450052 699 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:04:12.451082 699 webserver.cc:460] Webserver started at http://127.15.139.130:39181/ using document root <none> and password file <none>
I20260923 18:04:12.451804 699 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:12.451901 699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:12.452144 699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:12.454737 699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "a452791d41a745b590808031a45421e1"
format_stamp: "Formatted at 2026-09-23 18:04:12 on dist-test-slave-vr57"
server_key: "817337d17e14e758822c9ec907d5e626"
server_key_iv: "cb5ff7d268924b2c6f3094f180372861"
server_key_version: "encryptionkey@0"
I20260923 18:04:12.455336 699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "a452791d41a745b590808031a45421e1"
format_stamp: "Formatted at 2026-09-23 18:04:12 on dist-test-slave-vr57"
server_key: "817337d17e14e758822c9ec907d5e626"
server_key_iv: "cb5ff7d268924b2c6f3094f180372861"
server_key_version: "encryptionkey@0"
I20260923 18:04:12.460722 699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260923 18:04:12.464017 714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:12.465355 699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:04:12.465503 699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "a452791d41a745b590808031a45421e1"
format_stamp: "Formatted at 2026-09-23 18:04:12 on dist-test-slave-vr57"
server_key: "817337d17e14e758822c9ec907d5e626"
server_key_iv: "cb5ff7d268924b2c6f3094f180372861"
server_key_version: "encryptionkey@0"
I20260923 18:04:12.465637 699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:12.485078 699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:12.488808 699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:12.489053 699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:12.489890 699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:12.490821 699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:12.490892 699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:12.490974 699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:12.491005 699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:12.499699 699 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:44243
I20260923 18:04:12.499750 827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:44243 every 8 connection(s)
I20260923 18:04:12.500684 699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:04:12.504689 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 699
I20260923 18:04:12.504832 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:04:12.505210 15918 external_mini_cluster.cc:1428] Setting key ab591dfb543ecd72a806b4e32dffcc0c
Sep 23 18:04:12 dist-test-slave-vr57 krb5kdc[458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186652, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
I20260923 18:04:12.520563 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.502745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:40103 (local address 127.15.139.190:46077)
0923 18:04:12.502887 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:12.502892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:12.506426 (+ 3534us) server_negotiation.cc:407] Connection header received
0923 18:04:12.508494 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:12.508500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:12.508569 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:12.508667 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:12.510331 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.511225 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.513127 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.513345 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.513429 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.516741 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:12.516746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:12.516749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:12.516783 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:12.518217 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:12.518842 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:12.518845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:12.518846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:12.518922 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:12.519336 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:12.519345 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:12.519347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:12.519614 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:12.519726 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:12.520246 (+ 520us) server_negotiation.cc:299] Negotiation successful
0923 18:04:12.520380 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260923 18:04:12.521410 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.503298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:46077 (local address 127.15.139.130:40103)
0923 18:04:12.506251 (+ 2953us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:12.506316 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:12.508225 (+ 1909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:12.508870 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:12.508881 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:12.509413 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:12.510163 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:12.510185 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.511383 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.511387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:12.512993 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:12.513008 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.513516 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.513522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:12.513782 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.514462 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:12.514483 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:12.516421 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:12.518324 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:12.518329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:12.518342 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:12.518741 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:12.518998 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:12.519000 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:12.519002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:12.519234 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:12.519724 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:12.519732 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:12.519989 (+ 257us) client_negotiation.cc:769] Sending connection context
0923 18:04:12.520186 (+ 197us) client_negotiation.cc:240] Negotiation successful
0923 18:04:12.520526 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2669,"thread_start_us":197,"threads_started":1}
I20260923 18:04:12.522706 828 heartbeater.cc:347] Connected to a master server at 127.15.139.190:46077
I20260923 18:04:12.522985 828 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:12.523685 828 heartbeater.cc:511] Master 127.15.139.190:46077 requested a full tablet report, sending...
I20260923 18:04:12.524842 503 ts_manager.cc:194] Registered new tserver with Master: a452791d41a745b590808031a45421e1 (127.15.139.130:44243)
I20260923 18:04:12.525429 503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:40103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:04:12.536487 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:46077
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:12.652371 833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:12.652684 833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:12.652768 833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:12.656889 833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:12.656986 833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:12.657074 833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:04:12.662532 833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:12.663753 833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:12.664847 833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:12.674862 841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:12.674842 839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:12.674844 838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:12.674988 833 server_base.cc:1034] running on GCE node
I20260923 18:04:12.675673 833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:12.676362 833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:12.677568 833 hybrid_clock.cc:648] HybridClock initialized: now 1790186652677548 us; error 36 us; skew 500 ppm
Sep 23 18:04:12 dist-test-slave-vr57 krb5kdc[458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186652, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:12.680747 833 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:04:12.681650 833 webserver.cc:460] Webserver started at http://127.15.139.131:38511/ using document root <none> and password file <none>
I20260923 18:04:12.682228 833 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:12.682293 833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:12.682554 833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:12.684878 833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "18f817656d2e41c689ef6ec5fb5ce620"
format_stamp: "Formatted at 2026-09-23 18:04:12 on dist-test-slave-vr57"
server_key: "e9b2cc75ecc5d5773d1353a7dd8721ad"
server_key_iv: "5dd4765079168b66cc3fbc39fa6fc1e2"
server_key_version: "encryptionkey@0"
I20260923 18:04:12.685447 833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "18f817656d2e41c689ef6ec5fb5ce620"
format_stamp: "Formatted at 2026-09-23 18:04:12 on dist-test-slave-vr57"
server_key: "e9b2cc75ecc5d5773d1353a7dd8721ad"
server_key_iv: "5dd4765079168b66cc3fbc39fa6fc1e2"
server_key_version: "encryptionkey@0"
I20260923 18:04:12.690258 833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 18:04:12.693111 848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:12.694492 833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:04:12.694602 833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "18f817656d2e41c689ef6ec5fb5ce620"
format_stamp: "Formatted at 2026-09-23 18:04:12 on dist-test-slave-vr57"
server_key: "e9b2cc75ecc5d5773d1353a7dd8721ad"
server_key_iv: "5dd4765079168b66cc3fbc39fa6fc1e2"
server_key_version: "encryptionkey@0"
I20260923 18:04:12.694700 833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:12.712242 833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:12.715703 833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:12.715863 833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:12.716418 833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:12.717254 833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:12.717303 833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:12.717343 833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:12.717360 833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:12.726712 833 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:39133
I20260923 18:04:12.726732 961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:39133 every 8 connection(s)
I20260923 18:04:12.727787 833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:04:12.733484 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 833
I20260923 18:04:12.733667 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:04:12.733985 15918 external_mini_cluster.cc:1428] Setting key c398e65fc6efff5d1739798df7ad0b87
Sep 23 18:04:12 dist-test-slave-vr57 krb5kdc[458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186652, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:12.743021 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.729692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:47629 (local address 127.15.139.190:46077)
0923 18:04:12.729873 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:12.729877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:12.730758 (+ 881us) server_negotiation.cc:407] Connection header received
0923 18:04:12.732320 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:12.732323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:12.732382 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:12.732494 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:12.733673 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.734328 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.735831 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.736194 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.736312 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.738998 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:12.739001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:12.739003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:12.739027 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:12.740695 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:12.741247 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:12.741251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:12.741252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:12.741355 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:12.741783 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:12.741786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:12.741787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:12.741996 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:12.742091 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:12.742775 (+ 684us) server_negotiation.cc:299] Negotiation successful
0923 18:04:12.742889 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 18:04:12.743971 964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.730114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:46077 (local address 127.15.139.131:47629)
0923 18:04:12.730590 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:12.730636 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:12.732099 (+ 1463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:12.732671 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:12.732679 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:12.733039 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:12.733528 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:12.733541 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.734471 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.734475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:12.735715 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:12.735725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.736385 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.736391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:12.736599 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.737145 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:12.737162 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:12.738801 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:12.740800 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:12.740805 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:12.740819 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:12.741116 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:12.741532 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:12.741534 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:12.741535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:12.741672 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:12.742104 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:12.742109 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:12.742357 (+ 248us) client_negotiation.cc:769] Sending connection context
0923 18:04:12.742529 (+ 172us) client_negotiation.cc:240] Negotiation successful
0923 18:04:12.742813 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":120,"threads_started":1}
I20260923 18:04:12.745340 962 heartbeater.cc:347] Connected to a master server at 127.15.139.190:46077
I20260923 18:04:12.745653 962 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:12.746253 962 heartbeater.cc:511] Master 127.15.139.190:46077 requested a full tablet report, sending...
I20260923 18:04:12.747642 502 ts_manager.cc:194] Registered new tserver with Master: 18f817656d2e41c689ef6ec5fb5ce620 (127.15.139.131:39133)
I20260923 18:04:12.748394 502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:47629
I20260923 18:04:12.748631 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:04:12.758522 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.750841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33240 (local address 127.15.139.190:46077)
0923 18:04:12.750989 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:12.750994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:12.751200 (+ 206us) server_negotiation.cc:407] Connection header received
0923 18:04:12.751284 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:12.751286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:12.751326 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:12.751399 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:12.752219 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.752834 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.753688 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.753911 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.754007 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.755389 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:12.755392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:12.755394 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:12.755415 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:12.756976 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:12.757362 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:12.757365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:12.757366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:12.757439 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:12.757681 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:12.757683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:12.757684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:12.757886 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:12.757992 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:12.758206 (+ 214us) server_negotiation.cc:299] Negotiation successful
0923 18:04:12.758302 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 18:04:12.761705 502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:04:12.764717 502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:04:12.779958 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.774416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38552 (local address 127.15.139.129:42025)
0923 18:04:12.775067 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:12.775081 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:12.775100 (+ 19us) server_negotiation.cc:407] Connection header received
0923 18:04:12.775162 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:12.775166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:12.775307 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:12.775659 (+ 352us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:12.775836 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.776587 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.778168 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.779095 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.779212 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.779339 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:12.779682 (+ 343us) server_negotiation.cc:299] Negotiation successful
0923 18:04:12.779797 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":119,"threads_started":1}
I20260923 18:04:12.779978 973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.774222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:42025 (local address 127.0.0.1:38552)
0923 18:04:12.774741 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:12.774756 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:12.774829 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:12.775457 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:12.775460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:12.775477 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:12.775727 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:12.775737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.776728 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.776734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:12.778031 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:12.778041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.779342 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.779346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:12.779467 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.779534 (+ 67us) client_negotiation.cc:769] Sending connection context
0923 18:04:12.779648 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 18:04:12.779793 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":44,"spinlock_wait_cycles":896,"thread_start_us":127,"threads_started":1}
I20260923 18:04:12.780313 977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.774331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56704 (local address 127.15.139.131:39133)
0923 18:04:12.774918 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:12.774932 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:12.774955 (+ 23us) server_negotiation.cc:407] Connection header received
0923 18:04:12.775032 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:12.775037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:12.775263 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:12.775456 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:12.775862 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.777032 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.778360 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.779251 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.779363 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.779520 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:12.780075 (+ 555us) server_negotiation.cc:299] Negotiation successful
0923 18:04:12.780177 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":149,"threads_started":1}
I20260923 18:04:12.780565 974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.773417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:39133 (local address 127.0.0.1:56704)
0923 18:04:12.773947 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:12.773995 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:12.774137 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:12.775435 (+ 1298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:12.775441 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:12.775465 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:12.775725 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:12.775737 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.777190 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.777194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:12.778249 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:12.778256 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.779579 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.779582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:12.779840 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.779861 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 18:04:12.779918 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 18:04:12.779987 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":47,"thread_start_us":127,"threads_started":1}
I20260923 18:04:12.782511 976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.773534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:44243 (local address 127.0.0.1:36172)
0923 18:04:12.774555 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:12.774570 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:12.774665 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:12.775807 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:12.775809 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:12.775829 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:12.775913 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:12.775920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.778543 (+ 2623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.778546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:12.779692 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:12.779699 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.782056 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.782060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:12.782154 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.782177 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 18:04:12.782255 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 18:04:12.782326 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"mutex_wait_us":16,"thread_start_us":142,"threads_started":1}
I20260923 18:04:12.783061 975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.773718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36172 (local address 127.15.139.130:44243)
0923 18:04:12.775017 (+ 1299us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:12.775032 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:12.775055 (+ 23us) server_negotiation.cc:407] Connection header received
0923 18:04:12.775406 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:12.775412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:12.775643 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:12.776050 (+ 407us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:12.776205 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.777271 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.779793 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:12.781666 (+ 1873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:12.781796 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:12.781919 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:12.782810 (+ 891us) server_negotiation.cc:299] Negotiation successful
0923 18:04:12.782916 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1190,"thread_start_us":109,"threads_started":1}
I20260923 18:04:12.783452 628 tablet_service.cc:1467] Processing CreateTablet for tablet 2e5106d533904fe18e11d4f21ea899b3 (DEFAULT_TABLE table=test-table [id=1d62b38be4f64e8181d27f1d0ff3d6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:12.783437 896 tablet_service.cc:1467] Processing CreateTablet for tablet 2e5106d533904fe18e11d4f21ea899b3 (DEFAULT_TABLE table=test-table [id=1d62b38be4f64e8181d27f1d0ff3d6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:12.784466 628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e5106d533904fe18e11d4f21ea899b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:12.784523 896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e5106d533904fe18e11d4f21ea899b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:12.785830 762 tablet_service.cc:1467] Processing CreateTablet for tablet 2e5106d533904fe18e11d4f21ea899b3 (DEFAULT_TABLE table=test-table [id=1d62b38be4f64e8181d27f1d0ff3d6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:12.786849 762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e5106d533904fe18e11d4f21ea899b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:12.791471 979 tablet_bootstrap.cc:492] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98: Bootstrap starting.
I20260923 18:04:12.792248 980 tablet_bootstrap.cc:492] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620: Bootstrap starting.
I20260923 18:04:12.793529 979 tablet_bootstrap.cc:654] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:12.793883 981 tablet_bootstrap.cc:492] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1: Bootstrap starting.
I20260923 18:04:12.794421 980 tablet_bootstrap.cc:654] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:12.794538 979 log.cc:824] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:12.795742 980 log.cc:824] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:12.796141 981 tablet_bootstrap.cc:654] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:12.796772 979 tablet_bootstrap.cc:492] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98: No bootstrap required, opened a new log
I20260923 18:04:12.797021 979 ts_tablet_manager.cc:1397] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260923 18:04:12.797379 981 log.cc:824] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:12.797439 980 tablet_bootstrap.cc:492] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620: No bootstrap required, opened a new log
I20260923 18:04:12.797701 980 ts_tablet_manager.cc:1397] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260923 18:04:12.799957 981 tablet_bootstrap.cc:492] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1: No bootstrap required, opened a new log
I20260923 18:04:12.800140 981 ts_tablet_manager.cc:1397] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260923 18:04:12.801106 980 raft_consensus.cc:348] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f817656d2e41c689ef6ec5fb5ce620" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 39133 } } peers { permanent_uuid: "a452791d41a745b590808031a45421e1" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44243 } } peers { permanent_uuid: "ba97fed20a42408281bdbdd6e0188d98" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42025 } }
I20260923 18:04:12.801343 980 raft_consensus.cc:374] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:12.801414 980 raft_consensus.cc:729] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f817656d2e41c689ef6ec5fb5ce620, State: Initialized, Role: FOLLOWER
I20260923 18:04:12.801596 979 raft_consensus.cc:348] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f817656d2e41c689ef6ec5fb5ce620" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 39133 } } peers { permanent_uuid: "a452791d41a745b590808031a45421e1" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44243 } } peers { permanent_uuid: "ba97fed20a42408281bdbdd6e0188d98" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42025 } }
I20260923 18:04:12.801839 979 raft_consensus.cc:374] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:12.801904 979 raft_consensus.cc:729] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba97fed20a42408281bdbdd6e0188d98, State: Initialized, Role: FOLLOWER
I20260923 18:04:12.802018 980 consensus_queue.cc:260] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f817656d2e41c689ef6ec5fb5ce620" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 39133 } } peers { permanent_uuid: "a452791d41a745b590808031a45421e1" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44243 } } peers { permanent_uuid: "ba97fed20a42408281bdbdd6e0188d98" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42025 } }
I20260923 18:04:12.802362 979 consensus_queue.cc:260] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f817656d2e41c689ef6ec5fb5ce620" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 39133 } } peers { permanent_uuid: "a452791d41a745b590808031a45421e1" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44243 } } peers { permanent_uuid: "ba97fed20a42408281bdbdd6e0188d98" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42025 } }
I20260923 18:04:12.802701 962 heartbeater.cc:503] Master 127.15.139.190:46077 was elected leader, sending a full tablet report...
I20260923 18:04:12.803054 980 ts_tablet_manager.cc:1428] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260923 18:04:12.803957 981 raft_consensus.cc:348] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f817656d2e41c689ef6ec5fb5ce620" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 39133 } } peers { permanent_uuid: "a452791d41a745b590808031a45421e1" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44243 } } peers { permanent_uuid: "ba97fed20a42408281bdbdd6e0188d98" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42025 } }
I20260923 18:04:12.804266 981 raft_consensus.cc:374] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:12.804361 981 raft_consensus.cc:729] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a452791d41a745b590808031a45421e1, State: Initialized, Role: FOLLOWER
I20260923 18:04:12.804390 694 heartbeater.cc:503] Master 127.15.139.190:46077 was elected leader, sending a full tablet report...
I20260923 18:04:12.804466 979 ts_tablet_manager.cc:1428] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260923 18:04:12.804909 981 consensus_queue.cc:260] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f817656d2e41c689ef6ec5fb5ce620" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 39133 } } peers { permanent_uuid: "a452791d41a745b590808031a45421e1" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44243 } } peers { permanent_uuid: "ba97fed20a42408281bdbdd6e0188d98" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42025 } }
I20260923 18:04:12.805543 828 heartbeater.cc:503] Master 127.15.139.190:46077 was elected leader, sending a full tablet report...
I20260923 18:04:12.806694 981 ts_tablet_manager.cc:1428] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260923 18:04:12.978802 963 tablet.cc:2369] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:04:12.997022 986 raft_consensus.cc:482] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:04:12.997263 986 raft_consensus.cc:504] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f817656d2e41c689ef6ec5fb5ce620" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 39133 } } peers { permanent_uuid: "a452791d41a745b590808031a45421e1" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44243 } } peers { permanent_uuid: "ba97fed20a42408281bdbdd6e0188d98" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42025 } }
I20260923 18:04:12.998526 986 leader_election.cc:288] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18f817656d2e41c689ef6ec5fb5ce620 (127.15.139.131:39133), a452791d41a745b590808031a45421e1 (127.15.139.130:44243)
W20260923 18:04:13.001907 829 tablet.cc:2369] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:04:13.004247 977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.998897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:58347 (local address 127.15.139.131:39133)
0923 18:04:12.999096 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:12.999101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:12.999409 (+ 308us) server_negotiation.cc:407] Connection header received
0923 18:04:12.999614 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:12.999616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:12.999673 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:12.999762 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:13.000197 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.001038 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.002821 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.003530 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.003647 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:13.003700 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:13.004038 (+ 338us) server_negotiation.cc:299] Negotiation successful
0923 18:04:13.004114 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260923 18:04:13.004247 975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.998904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:40753 (local address 127.15.139.130:44243)
0923 18:04:12.999096 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:12.999101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:12.999413 (+ 312us) server_negotiation.cc:407] Connection header received
0923 18:04:12.999611 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:12.999615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:12.999672 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:12.999760 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:13.000196 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.001021 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.002821 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.003482 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.003647 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:13.003700 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:13.004038 (+ 338us) server_negotiation.cc:299] Negotiation successful
0923 18:04:13.004114 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260923 18:04:13.004338 988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.998819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:44243 (local address 127.15.139.129:40753)
0923 18:04:12.999317 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:12.999332 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:12.999454 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:12.999790 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:12.999793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:12.999821 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:13.000065 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:13.000073 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.001153 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.001157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:13.002670 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:13.002683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.003652 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.003656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:13.003869 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:13.003922 (+ 53us) client_negotiation.cc:769] Sending connection context
0923 18:04:13.004007 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 18:04:13.004120 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":244,"spinlock_wait_cycles":30720,"thread_start_us":101,"threads_started":1}
I20260923 18:04:13.004338 989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:12.998816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:39133 (local address 127.15.139.129:58347)
0923 18:04:12.999291 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:12.999306 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:12.999454 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:12.999790 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:12.999793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:12.999820 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:13.000065 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:13.000073 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.001153 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.001157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:13.002672 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:13.002683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.003652 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.003656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:13.003865 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:13.003922 (+ 57us) client_negotiation.cc:769] Sending connection context
0923 18:04:13.004007 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 18:04:13.004120 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":234,"spinlock_wait_cycles":17408,"thread_start_us":166,"threads_started":1}
I20260923 18:04:13.004998 782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e5106d533904fe18e11d4f21ea899b3" candidate_uuid: "ba97fed20a42408281bdbdd6e0188d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a452791d41a745b590808031a45421e1" is_pre_election: true
I20260923 18:04:13.005024 916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e5106d533904fe18e11d4f21ea899b3" candidate_uuid: "ba97fed20a42408281bdbdd6e0188d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f817656d2e41c689ef6ec5fb5ce620" is_pre_election: true
I20260923 18:04:13.005313 916 raft_consensus.cc:2454] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba97fed20a42408281bdbdd6e0188d98 in term 0.
I20260923 18:04:13.005321 782 raft_consensus.cc:2454] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba97fed20a42408281bdbdd6e0188d98 in term 0.
I20260923 18:04:13.005880 582 leader_election.cc:302] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 18f817656d2e41c689ef6ec5fb5ce620, a452791d41a745b590808031a45421e1, ba97fed20a42408281bdbdd6e0188d98; no voters:
I20260923 18:04:13.006191 986 raft_consensus.cc:2781] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:04:13.006269 986 raft_consensus.cc:482] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:04:13.006323 986 raft_consensus.cc:3037] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:13.007400 986 raft_consensus.cc:504] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f817656d2e41c689ef6ec5fb5ce620" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 39133 } } peers { permanent_uuid: "a452791d41a745b590808031a45421e1" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44243 } } peers { permanent_uuid: "ba97fed20a42408281bdbdd6e0188d98" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42025 } }
I20260923 18:04:13.007814 986 leader_election.cc:288] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [CANDIDATE]: Term 1 election: Requested vote from peers 18f817656d2e41c689ef6ec5fb5ce620 (127.15.139.131:39133), a452791d41a745b590808031a45421e1 (127.15.139.130:44243)
I20260923 18:04:13.008179 782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e5106d533904fe18e11d4f21ea899b3" candidate_uuid: "ba97fed20a42408281bdbdd6e0188d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a452791d41a745b590808031a45421e1"
I20260923 18:04:13.008174 916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e5106d533904fe18e11d4f21ea899b3" candidate_uuid: "ba97fed20a42408281bdbdd6e0188d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f817656d2e41c689ef6ec5fb5ce620"
I20260923 18:04:13.008314 916 raft_consensus.cc:3037] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:13.008285 782 raft_consensus.cc:3037] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:13.009359 782 raft_consensus.cc:2454] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba97fed20a42408281bdbdd6e0188d98 in term 1.
I20260923 18:04:13.009433 916 raft_consensus.cc:2454] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba97fed20a42408281bdbdd6e0188d98 in term 1.
I20260923 18:04:13.009683 583 leader_election.cc:302] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a452791d41a745b590808031a45421e1, ba97fed20a42408281bdbdd6e0188d98; no voters:
I20260923 18:04:13.009853 986 raft_consensus.cc:2781] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:13.010128 986 raft_consensus.cc:686] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [term 1 LEADER]: Becoming Leader. State: Replica: ba97fed20a42408281bdbdd6e0188d98, State: Running, Role: LEADER
I20260923 18:04:13.010475 986 consensus_queue.cc:237] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f817656d2e41c689ef6ec5fb5ce620" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 39133 } } peers { permanent_uuid: "a452791d41a745b590808031a45421e1" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44243 } } peers { permanent_uuid: "ba97fed20a42408281bdbdd6e0188d98" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42025 } }
I20260923 18:04:13.014040 502 catalog_manager.cc:5495] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba97fed20a42408281bdbdd6e0188d98 (127.15.139.129). New cstate: current_term: 1 leader_uuid: "ba97fed20a42408281bdbdd6e0188d98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f817656d2e41c689ef6ec5fb5ce620" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 39133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a452791d41a745b590808031a45421e1" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba97fed20a42408281bdbdd6e0188d98" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 42025 } health_report { overall_health: HEALTHY } } }
W20260923 18:04:13.024730 695 tablet.cc:2369] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:04:13.077320 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:13.073768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38562 (local address 127.15.139.129:42025)
0923 18:04:13.073893 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:13.073897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:13.073912 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:04:13.074055 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:13.074057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:13.074118 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:13.074217 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:13.074432 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.075013 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.075927 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.076166 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.076258 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:13.076562 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:13.076991 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:13.077089 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:13.077147 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 18:04:13.077200 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 18:04:13.083611 916 raft_consensus.cc:1261] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620 [term 1 FOLLOWER]: Refusing update from remote peer ba97fed20a42408281bdbdd6e0188d98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:13.083623 782 raft_consensus.cc:1261] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1 [term 1 FOLLOWER]: Refusing update from remote peer ba97fed20a42408281bdbdd6e0188d98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:13.084342 990 consensus_queue.cc:1035] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f817656d2e41c689ef6ec5fb5ce620" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 39133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:13.084558 986 consensus_queue.cc:1035] T 2e5106d533904fe18e11d4f21ea899b3 P ba97fed20a42408281bdbdd6e0188d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a452791d41a745b590808031a45421e1" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:13.090962 993 mvcc.cc:205] Tried to move back new op lower bound from 7332604531022491648 to 7332604530734268416. Current Snapshot: MvccSnapshot[applied={T|T < 7332604531022491648}]
I20260923 18:04:13.091530 994 mvcc.cc:205] Tried to move back new op lower bound from 7332604531022491648 to 7332604530734268416. Current Snapshot: MvccSnapshot[applied={T|T < 7332604531022491648}]
I20260923 18:04:13.097716 995 mvcc.cc:205] Tried to move back new op lower bound from 7332604531022491648 to 7332604530734268416. Current Snapshot: MvccSnapshot[applied={T|T < 7332604531022491648}]
I20260923 18:04:13.100116 502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:04:13.100283 502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:04:13.102632 502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 58b1fce3cbe74e8898a3f27fcbf971a3: Sending DeleteTablet for 3 replicas of tablet 2e5106d533904fe18e11d4f21ea899b3
I20260923 18:04:13.103443 762 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e5106d533904fe18e11d4f21ea899b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:13 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:36172
I20260923 18:04:13.103438 628 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e5106d533904fe18e11d4f21ea899b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:13 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:38552
I20260923 18:04:13.103703 896 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e5106d533904fe18e11d4f21ea899b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:13 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:56704
I20260923 18:04:13.103971 1003 tablet_replica.cc:330] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1: stopping tablet replica
I20260923 18:04:13.104182 1005 tablet_replica.cc:330] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620: stopping tablet replica
I20260923 18:04:13.104324 1003 raft_consensus.cc:2229] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:04:13.104672 1003 raft_consensus.cc:2258] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:13.104687 1005 raft_consensus.cc:2229] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:04:13.104941 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 565
I20260923 18:04:13.106288 1005 raft_consensus.cc:2258] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:13.107801 1005 ts_tablet_manager.cc:1905] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:04:13.110054 1005 ts_tablet_manager.cc:1918] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:04:13.110179 1005 log.cc:1196] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/2e5106d533904fe18e11d4f21ea899b3
I20260923 18:04:13.110523 1005 ts_tablet_manager.cc:1939] T 2e5106d533904fe18e11d4f21ea899b3 P 18f817656d2e41c689ef6ec5fb5ce620: Deleting consensus metadata
I20260923 18:04:13.111366 1003 ts_tablet_manager.cc:1905] T 2e5106d533904fe18e11d4f21ea899b3 P a452791d41a745b590808031a45421e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:04:13.111783 488 catalog_manager.cc:4841] TS 18f817656d2e41c689ef6ec5fb5ce620 (127.15.139.131:39133): tablet 2e5106d533904fe18e11d4f21ea899b3 (table test-table [id=1d62b38be4f64e8181d27f1d0ff3d6c8]) successfully deleted
W20260923 18:04:13.112509 488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:42025 (error 108)
I20260923 18:04:13.112802 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 699
I20260923 18:04:13.113875 976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:13.113589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:42025 (local address 127.0.0.1:38570)
0923 18:04:13.113719 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:13.113790 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:42025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260923 18:04:13.114029 488 catalog_manager.cc:4584] TS ba97fed20a42408281bdbdd6e0188d98 (127.15.139.129:42025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e5106d533904fe18e11d4f21ea899b3: Network error: Client connection negotiation failed: client connection to 127.15.139.129:42025: connect: Connection refused (error 111)
I20260923 18:04:13.119186 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 833
I20260923 18:04:13.119827 976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:13.119365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:44243 (local address 127.0.0.1:36174)
0923 18:04:13.119653 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:13.119719 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.130:44243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260923 18:04:13.124873 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 471
2026-09-23T18:04:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2565 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-23T18:04:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:13Z Disabled control of system clock
I20260923 18:04:13.156366 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36023
--webserver_interface=127.15.139.190
--webserver_port=0
--builtin_ntp_servers=127.15.139.148:42683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:36023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 18:04:13.267748 1010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:13.268055 1010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:13.268092 1010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:13.272099 1010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:13.272156 1010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:13.272172 1010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:13.272189 1010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:13.272204 1010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:13.276854 1010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:36023
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:13.277875 1010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:13.279403 1010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:13.287935 1018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:13.287995 1015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:13.287967 1016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:13.288110 1010 server_base.cc:1034] running on GCE node
I20260923 18:04:13.288717 1010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:13.289788 1010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:13.290963 1010 hybrid_clock.cc:648] HybridClock initialized: now 1790186653290930 us; error 42 us; skew 500 ppm
I20260923 18:04:13.293007 1010 webserver.cc:460] Webserver started at http://127.15.139.190:32839/ using document root <none> and password file <none>
I20260923 18:04:13.293596 1010 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:13.293659 1010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:13.293900 1010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:13.295992 1010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "957f7472b23241d683fe240ac536525a"
format_stamp: "Formatted at 2026-09-23 18:04:13 on dist-test-slave-vr57"
server_key: "c69960ee2ba483fbe99e338983f0af07"
server_key_iv: "d643a5b91f26029a8892c63e13228c54"
server_key_version: "encryptionkey@0"
I20260923 18:04:13.296562 1010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "957f7472b23241d683fe240ac536525a"
format_stamp: "Formatted at 2026-09-23 18:04:13 on dist-test-slave-vr57"
server_key: "c69960ee2ba483fbe99e338983f0af07"
server_key_iv: "d643a5b91f26029a8892c63e13228c54"
server_key_version: "encryptionkey@0"
I20260923 18:04:13.300199 1010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:13.302656 1024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:13.303916 1010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:04:13.304039 1010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "957f7472b23241d683fe240ac536525a"
format_stamp: "Formatted at 2026-09-23 18:04:13 on dist-test-slave-vr57"
server_key: "c69960ee2ba483fbe99e338983f0af07"
server_key_iv: "d643a5b91f26029a8892c63e13228c54"
server_key_version: "encryptionkey@0"
I20260923 18:04:13.304148 1010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:13.330329 1010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:13.331087 1010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:13.331301 1010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:13.338387 1010 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:36023
I20260923 18:04:13.338397 1076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:36023 every 8 connection(s)
I20260923 18:04:13.339352 1010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:13.342242 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 1010
I20260923 18:04:13.342386 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:13.342638 1077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:13.342732 15918 external_mini_cluster.cc:1428] Setting key ecb34ac4018ea9d1c3b419a3a9da852d
I20260923 18:04:13.349352 1077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a: Bootstrap starting.
I20260923 18:04:13.351945 1077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:13.352736 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:13.344137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33316 (local address 127.15.139.190:36023)
0923 18:04:13.344752 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:13.344768 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:13.344802 (+ 34us) server_negotiation.cc:407] Connection header received
0923 18:04:13.345760 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:13.345779 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:13.346190 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:13.346471 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:13.346865 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.347933 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.349149 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.349664 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.349788 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:13.350657 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:13.350661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:13.350674 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:13.351075 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:13.351131 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:13.351150 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:13.351228 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:13.351382 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 18:04:13.351694 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":143,"threads_started":1}
I20260923 18:04:13.352936 1077 log.cc:824] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:13.355206 1077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a: No bootstrap required, opened a new log
I20260923 18:04:13.359063 1077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957f7472b23241d683fe240ac536525a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36023 } }
I20260923 18:04:13.359313 1077 raft_consensus.cc:374] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:13.359393 1077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 957f7472b23241d683fe240ac536525a, State: Initialized, Role: FOLLOWER
I20260923 18:04:13.359858 1077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957f7472b23241d683fe240ac536525a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36023 } }
I20260923 18:04:13.360030 1077 raft_consensus.cc:388] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:13.360149 1077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:13.360251 1077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:13.361174 1077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957f7472b23241d683fe240ac536525a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36023 } }
I20260923 18:04:13.361518 1077 leader_election.cc:302] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 957f7472b23241d683fe240ac536525a; no voters:
I20260923 18:04:13.361845 1077 leader_election.cc:288] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:13.362063 1082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:13.362317 1082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [term 1 LEADER]: Becoming Leader. State: Replica: 957f7472b23241d683fe240ac536525a, State: Running, Role: LEADER
I20260923 18:04:13.362643 1082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957f7472b23241d683fe240ac536525a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36023 } }
I20260923 18:04:13.363262 1077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:13.364246 1083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "957f7472b23241d683fe240ac536525a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957f7472b23241d683fe240ac536525a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36023 } } }
I20260923 18:04:13.364284 1084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 957f7472b23241d683fe240ac536525a. Latest consensus state: current_term: 1 leader_uuid: "957f7472b23241d683fe240ac536525a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957f7472b23241d683fe240ac536525a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36023 } } }
I20260923 18:04:13.364368 1083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:13.364396 1084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:13.364771 1091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:13.367903 1091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:13.373335 1091 catalog_manager.cc:1347] Generated new cluster ID: de4a9daf7e3541ab8a7aec3b4ff2bcf0
I20260923 18:04:13.373409 1091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:13.393851 1091 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:13.395205 1091 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:13.401989 1091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a: Generated new TSK 0
I20260923 18:04:13.402895 1091 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:13.416241 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36023
--builtin_ntp_servers=127.15.139.148:42683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 18:04:13.528402 1101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:13.528718 1101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:13.528792 1101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:13.532821 1101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:13.532912 1101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:13.533005 1101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:04:13.537829 1101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:13.538956 1101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:13.540469 1101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:13.550450 1107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:13.550505 1106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:13.550455 1109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:13.550917 1101 server_base.cc:1034] running on GCE node
I20260923 18:04:13.551276 1101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:13.551957 1101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:13.553117 1101 hybrid_clock.cc:648] HybridClock initialized: now 1790186653553101 us; error 52 us; skew 500 ppm
I20260923 18:04:13.554986 1101 webserver.cc:460] Webserver started at http://127.15.139.129:37179/ using document root <none> and password file <none>
I20260923 18:04:13.555626 1101 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:13.555694 1101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:13.555913 1101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:13.557938 1101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "b14e620beab140298ae018d7de2b3769"
format_stamp: "Formatted at 2026-09-23 18:04:13 on dist-test-slave-vr57"
server_key: "aa5497071a3de45d342f1423df3b71d3"
server_key_iv: "774f67741afa438c4eb69f77fafa0ffc"
server_key_version: "encryptionkey@0"
I20260923 18:04:13.558499 1101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "b14e620beab140298ae018d7de2b3769"
format_stamp: "Formatted at 2026-09-23 18:04:13 on dist-test-slave-vr57"
server_key: "aa5497071a3de45d342f1423df3b71d3"
server_key_iv: "774f67741afa438c4eb69f77fafa0ffc"
server_key_version: "encryptionkey@0"
I20260923 18:04:13.562171 1101 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260923 18:04:13.564916 1115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:13.566172 1101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:04:13.566308 1101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "b14e620beab140298ae018d7de2b3769"
format_stamp: "Formatted at 2026-09-23 18:04:13 on dist-test-slave-vr57"
server_key: "aa5497071a3de45d342f1423df3b71d3"
server_key_iv: "774f67741afa438c4eb69f77fafa0ffc"
server_key_version: "encryptionkey@0"
I20260923 18:04:13.566417 1101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:13.578688 1101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:13.579442 1101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:13.579672 1101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:13.580255 1101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:13.581198 1101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:13.581267 1101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:13.581332 1101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:13.581383 1101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:13.590776 1101 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:37321
I20260923 18:04:13.590807 1228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:37321 every 8 connection(s)
I20260923 18:04:13.591794 1101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:04:13.593191 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 1101
I20260923 18:04:13.593340 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:04:13.593695 15918 external_mini_cluster.cc:1428] Setting key 807ebd2d3017ce771e053e09f5115bf9
I20260923 18:04:13.595813 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36023
--builtin_ntp_servers=127.15.139.148:42683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 18:04:13.602259 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:13.593832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:44487 (local address 127.15.139.190:36023)
0923 18:04:13.594011 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:13.594016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:13.595185 (+ 1169us) server_negotiation.cc:407] Connection header received
0923 18:04:13.596725 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:13.596732 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:13.596793 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:13.596893 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:13.598193 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.598738 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.599999 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.600397 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.600547 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:13.601652 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:13.601659 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:13.601664 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:13.601683 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:13.601711 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:13.601726 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:13.601817 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:13.601961 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 18:04:13.602044 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 18:04:13.603008 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:13.594264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36023 (local address 127.15.139.129:44487)
0923 18:04:13.595019 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:13.595097 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:13.596510 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:13.597137 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:13.597149 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:13.597273 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:13.598016 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:13.598039 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.598859 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.598861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:13.599848 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:13.599858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.600564 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.600573 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:13.600717 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:13.601383 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:04:13.601384 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:13.601424 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:04:13.601488 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:13.601820 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:13.601827 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:13.601831 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 18:04:13.601925 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 18:04:13.602162 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":156,"threads_started":1}
I20260923 18:04:13.604370 1229 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36023
I20260923 18:04:13.604604 1229 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:13.605146 1229 heartbeater.cc:511] Master 127.15.139.190:36023 requested a full tablet report, sending...
I20260923 18:04:13.606745 1041 ts_manager.cc:194] Registered new tserver with Master: b14e620beab140298ae018d7de2b3769 (127.15.139.129:37321)
I20260923 18:04:13.608018 1041 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.139.129:44487
W20260923 18:04:13.714265 1232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:13.714654 1232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:13.714738 1232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:13.718885 1232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:13.718956 1232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:13.719067 1232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:04:13.723965 1232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:13.725149 1232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:13.726585 1232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:13.736114 1238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:13.736135 1240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:13.736136 1237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:13.736517 1232 server_base.cc:1034] running on GCE node
I20260923 18:04:13.736925 1232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:13.737610 1232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:13.738797 1232 hybrid_clock.cc:648] HybridClock initialized: now 1790186653738774 us; error 39 us; skew 500 ppm
I20260923 18:04:13.740772 1232 webserver.cc:460] Webserver started at http://127.15.139.130:34365/ using document root <none> and password file <none>
I20260923 18:04:13.741360 1232 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:13.741456 1232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:13.741691 1232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:13.743871 1232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "1facf99c73cf45caa6b54d69c469f1e9"
format_stamp: "Formatted at 2026-09-23 18:04:13 on dist-test-slave-vr57"
server_key: "828554cc15073abe53c8dd61fbd5ea3e"
server_key_iv: "c9e8bebf5d05fcce1f440e35171df3e0"
server_key_version: "encryptionkey@0"
I20260923 18:04:13.744445 1232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "1facf99c73cf45caa6b54d69c469f1e9"
format_stamp: "Formatted at 2026-09-23 18:04:13 on dist-test-slave-vr57"
server_key: "828554cc15073abe53c8dd61fbd5ea3e"
server_key_iv: "c9e8bebf5d05fcce1f440e35171df3e0"
server_key_version: "encryptionkey@0"
I20260923 18:04:13.748283 1232 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260923 18:04:13.750819 1246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:13.752326 1232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:04:13.752463 1232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "1facf99c73cf45caa6b54d69c469f1e9"
format_stamp: "Formatted at 2026-09-23 18:04:13 on dist-test-slave-vr57"
server_key: "828554cc15073abe53c8dd61fbd5ea3e"
server_key_iv: "c9e8bebf5d05fcce1f440e35171df3e0"
server_key_version: "encryptionkey@0"
I20260923 18:04:13.752591 1232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:13.772229 1232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:13.772954 1232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:13.773164 1232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:13.773792 1232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:13.774745 1232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:13.774816 1232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:13.774891 1232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:13.774942 1232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:13.784948 1232 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:35541
I20260923 18:04:13.785013 1359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:35541 every 8 connection(s)
I20260923 18:04:13.785936 1232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:04:13.793602 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 1232
I20260923 18:04:13.793756 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:04:13.794106 15918 external_mini_cluster.cc:1428] Setting key a8af7ee63f2d109479e2f74bd1ffc014
I20260923 18:04:13.796065 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:13.788028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:40857 (local address 127.15.139.190:36023)
0923 18:04:13.788186 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:13.788190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:13.789333 (+ 1143us) server_negotiation.cc:407] Connection header received
0923 18:04:13.790726 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:13.790729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:13.790791 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:13.790879 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:13.791876 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.792448 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.793592 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.793969 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.794094 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:13.795480 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:13.795483 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:13.795485 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:13.795537 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:13.795566 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:13.795576 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:13.795641 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:13.795855 (+ 214us) server_negotiation.cc:299] Negotiation successful
0923 18:04:13.795913 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 18:04:13.796370 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36023
--builtin_ntp_servers=127.15.139.148:42683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 18:04:13.797123 1362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:13.788497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36023 (local address 127.15.139.130:40857)
0923 18:04:13.789164 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:13.789206 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:13.790492 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:13.791042 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:13.791049 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:13.791137 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:13.791745 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:13.791762 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.792584 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.792586 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:13.793459 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:13.793468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.794128 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.794131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:13.794327 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:13.795135 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:04:13.795137 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:13.795179 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:04:13.795247 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:13.795671 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:13.795681 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:13.795690 (+ 9us) client_negotiation.cc:769] Sending connection context
0923 18:04:13.795824 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 18:04:13.796061 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":121,"threads_started":1}
I20260923 18:04:13.798373 1360 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36023
I20260923 18:04:13.798640 1360 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:13.799206 1360 heartbeater.cc:511] Master 127.15.139.190:36023 requested a full tablet report, sending...
I20260923 18:04:13.800356 1041 ts_manager.cc:194] Registered new tserver with Master: 1facf99c73cf45caa6b54d69c469f1e9 (127.15.139.130:35541)
I20260923 18:04:13.800984 1041 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.139.130:40857
W20260923 18:04:13.912202 1363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:13.912501 1363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:13.912580 1363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:13.916787 1363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:13.916854 1363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:13.916915 1363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:04:13.921850 1363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:42683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:13.922988 1363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:13.924456 1363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:13.934549 1371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:13.934549 1368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:13.934552 1369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:13.934794 1363 server_base.cc:1034] running on GCE node
I20260923 18:04:13.935307 1363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:13.936004 1363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:13.937208 1363 hybrid_clock.cc:648] HybridClock initialized: now 1790186653937194 us; error 32 us; skew 500 ppm
I20260923 18:04:13.938985 1363 webserver.cc:460] Webserver started at http://127.15.139.131:32917/ using document root <none> and password file <none>
I20260923 18:04:13.939626 1363 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:13.939719 1363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:13.939926 1363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:13.941977 1363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "26e133bee67f4a01b82f0e12670cc763"
format_stamp: "Formatted at 2026-09-23 18:04:13 on dist-test-slave-vr57"
server_key: "a8bbc724aa51582391897dce412f62fe"
server_key_iv: "88758de9cb9257468202c4abc3420058"
server_key_version: "encryptionkey@0"
I20260923 18:04:13.942616 1363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "26e133bee67f4a01b82f0e12670cc763"
format_stamp: "Formatted at 2026-09-23 18:04:13 on dist-test-slave-vr57"
server_key: "a8bbc724aa51582391897dce412f62fe"
server_key_iv: "88758de9cb9257468202c4abc3420058"
server_key_version: "encryptionkey@0"
I20260923 18:04:13.946381 1363 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:04:13.948920 1377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:13.950189 1363 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:04:13.950331 1363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "26e133bee67f4a01b82f0e12670cc763"
format_stamp: "Formatted at 2026-09-23 18:04:13 on dist-test-slave-vr57"
server_key: "a8bbc724aa51582391897dce412f62fe"
server_key_iv: "88758de9cb9257468202c4abc3420058"
server_key_version: "encryptionkey@0"
I20260923 18:04:13.950466 1363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:13.966270 1363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:13.967010 1363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:13.967226 1363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:13.967941 1363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:13.968900 1363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:13.968956 1363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:13.969018 1363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:13.969051 1363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:13.978609 1363 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:45011
I20260923 18:04:13.978718 1490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:45011 every 8 connection(s)
I20260923 18:04:13.979633 1363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:04:13.983268 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 1363
I20260923 18:04:13.983368 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:04:13.983767 15918 external_mini_cluster.cc:1428] Setting key 8291ed0e807b7209bba357e46b0548d4
I20260923 18:04:13.989616 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:13.981560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:50649 (local address 127.15.139.190:36023)
0923 18:04:13.981701 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:13.981704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:13.982782 (+ 1078us) server_negotiation.cc:407] Connection header received
0923 18:04:13.984625 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:13.984629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:13.984695 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:13.984792 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:13.985957 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.986486 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.987770 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.987983 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.988072 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:13.989156 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:13.989159 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:13.989160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:13.989173 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:13.989192 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:13.989199 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:13.989259 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:13.989443 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 18:04:13.989495 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 18:04:13.990585 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:13.981921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36023 (local address 127.15.139.131:50649)
0923 18:04:13.982624 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:13.982666 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:13.984355 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:13.985026 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:13.985044 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:13.985160 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:13.985772 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:13.985786 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.986601 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.986603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:13.987665 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:13.987675 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:13.988114 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:13.988116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:13.988252 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:13.988904 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:04:13.988905 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:13.988951 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:04:13.989019 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:13.989300 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:13.989308 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:13.989313 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 18:04:13.989442 (+ 129us) client_negotiation.cc:240] Negotiation successful
0923 18:04:13.989679 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":120,"threads_started":1}
I20260923 18:04:13.991792 1491 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36023
I20260923 18:04:13.992023 1491 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:13.992550 1491 heartbeater.cc:511] Master 127.15.139.190:36023 requested a full tablet report, sending...
I20260923 18:04:13.993584 1041 ts_manager.cc:194] Registered new tserver with Master: 26e133bee67f4a01b82f0e12670cc763 (127.15.139.131:45011)
I20260923 18:04:13.994112 1041 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.139.131:50649
I20260923 18:04:13.998010 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:04:14.003486 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.000052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33318 (local address 127.15.139.190:36023)
0923 18:04:14.000172 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:14.000176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:14.000338 (+ 162us) server_negotiation.cc:407] Connection header received
0923 18:04:14.000439 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:14.000442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:14.000483 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:14.000553 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:14.000826 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.001323 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.002143 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.002355 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.002438 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:14.003165 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:14.003168 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:14.003170 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:14.003181 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:14.003198 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:14.003206 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:14.003262 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:14.003334 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 18:04:14.003379 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 18:04:14.006502 1041 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:04:14.009775 1041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:04:14.024242 1505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.018129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:37321 (local address 127.0.0.1:56404)
0923 18:04:14.019811 (+ 1682us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:14.019823 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:14.019906 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:14.020447 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:14.020450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:14.020464 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:14.020552 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:14.020557 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.021577 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.021582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:14.022787 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:14.022796 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.023677 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.023680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:14.023786 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:14.023829 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 18:04:14.023945 (+ 116us) client_negotiation.cc:240] Negotiation successful
0923 18:04:14.024067 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1090,"mutex_wait_us":89,"thread_start_us":79,"threads_started":1}
I20260923 18:04:14.024276 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.018278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56404 (local address 127.15.139.129:37321)
0923 18:04:14.019238 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:14.019251 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:14.019922 (+ 671us) server_negotiation.cc:407] Connection header received
0923 18:04:14.020002 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:14.020006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:14.020176 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:14.020308 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:14.020646 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.021389 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.022923 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.023557 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.023683 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:14.023873 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:14.023992 (+ 119us) server_negotiation.cc:299] Negotiation successful
0923 18:04:14.024105 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838,"thread_start_us":133,"threads_started":1}
I20260923 18:04:14.024966 1503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.018501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:35541 (local address 127.0.0.1:37110)
0923 18:04:14.019420 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:14.019481 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:14.019631 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:14.020167 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:14.020173 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:14.020195 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:14.020336 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:14.020344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.021854 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.021857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:14.023352 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:14.023358 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.024526 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.024530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:14.024642 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:14.024667 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 18:04:14.024734 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 18:04:14.024823 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"mutex_wait_us":151,"thread_start_us":83,"threads_started":1}
I20260923 18:04:14.025128 1502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.018163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:45011 (local address 127.0.0.1:51426)
0923 18:04:14.019422 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:14.019484 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:14.019634 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:14.020326 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:14.020329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:14.020346 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:14.020452 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:14.020458 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.021854 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.021857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:14.023346 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:14.023358 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.024720 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.024726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:14.024833 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:14.024862 (+ 29us) client_negotiation.cc:769] Sending connection context
0923 18:04:14.024931 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 18:04:14.025007 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"mutex_wait_us":174,"thread_start_us":115,"threads_started":1}
I20260923 18:04:14.025604 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.018940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.15.139.131:45011)
0923 18:04:14.019417 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:14.019428 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:14.019605 (+ 177us) server_negotiation.cc:407] Connection header received
0923 18:04:14.019799 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:14.019805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:14.020016 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:14.020162 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:14.020570 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.021714 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.023534 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.024364 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.024491 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:14.024655 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:14.025348 (+ 693us) server_negotiation.cc:299] Negotiation successful
0923 18:04:14.025448 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":210,"threads_started":1}
I20260923 18:04:14.025915 1506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.018913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37110 (local address 127.15.139.130:35541)
0923 18:04:14.019665 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:14.019676 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:14.019702 (+ 26us) server_negotiation.cc:407] Connection header received
0923 18:04:14.019782 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:14.019787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:14.020008 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:14.020158 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:14.020462 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.021714 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.023484 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.024352 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.024477 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:14.025417 (+ 940us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:14.025473 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 18:04:14.025570 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":118,"threads_started":1}
I20260923 18:04:14.027791 1163 tablet_service.cc:1467] Processing CreateTablet for tablet 28c3e81ed15a4b54b6224372af2847fc (DEFAULT_TABLE table=test-table [id=2b1ecae678be4b22bf7dea63de8101e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:14.028399 1425 tablet_service.cc:1467] Processing CreateTablet for tablet 28c3e81ed15a4b54b6224372af2847fc (DEFAULT_TABLE table=test-table [id=2b1ecae678be4b22bf7dea63de8101e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:14.028841 1163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28c3e81ed15a4b54b6224372af2847fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:14.028401 1294 tablet_service.cc:1467] Processing CreateTablet for tablet 28c3e81ed15a4b54b6224372af2847fc (DEFAULT_TABLE table=test-table [id=2b1ecae678be4b22bf7dea63de8101e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:14.029454 1425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28c3e81ed15a4b54b6224372af2847fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:14.029456 1294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28c3e81ed15a4b54b6224372af2847fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:14.035364 1508 tablet_bootstrap.cc:492] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769: Bootstrap starting.
I20260923 18:04:14.037600 1509 tablet_bootstrap.cc:492] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763: Bootstrap starting.
I20260923 18:04:14.037694 1508 tablet_bootstrap.cc:654] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:14.038450 1508 log.cc:824] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:14.039453 1509 tablet_bootstrap.cc:654] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:14.039469 1510 tablet_bootstrap.cc:492] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9: Bootstrap starting.
I20260923 18:04:14.040264 1509 log.cc:824] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:14.040726 1508 tablet_bootstrap.cc:492] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769: No bootstrap required, opened a new log
I20260923 18:04:14.040925 1508 ts_tablet_manager.cc:1397] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260923 18:04:14.041688 1510 tablet_bootstrap.cc:654] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:14.042062 1509 tablet_bootstrap.cc:492] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763: No bootstrap required, opened a new log
I20260923 18:04:14.042263 1509 ts_tablet_manager.cc:1397] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260923 18:04:14.042505 1510 log.cc:824] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:14.044543 1510 tablet_bootstrap.cc:492] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9: No bootstrap required, opened a new log
I20260923 18:04:14.044731 1510 ts_tablet_manager.cc:1397] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260923 18:04:14.045373 1508 raft_consensus.cc:348] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14e620beab140298ae018d7de2b3769" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 37321 } } peers { permanent_uuid: "26e133bee67f4a01b82f0e12670cc763" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45011 } } peers { permanent_uuid: "1facf99c73cf45caa6b54d69c469f1e9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35541 } }
I20260923 18:04:14.045662 1508 raft_consensus.cc:374] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:14.045725 1508 raft_consensus.cc:729] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b14e620beab140298ae018d7de2b3769, State: Initialized, Role: FOLLOWER
I20260923 18:04:14.045699 1509 raft_consensus.cc:348] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14e620beab140298ae018d7de2b3769" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 37321 } } peers { permanent_uuid: "26e133bee67f4a01b82f0e12670cc763" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45011 } } peers { permanent_uuid: "1facf99c73cf45caa6b54d69c469f1e9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35541 } }
I20260923 18:04:14.045943 1509 raft_consensus.cc:374] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:14.046001 1509 raft_consensus.cc:729] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26e133bee67f4a01b82f0e12670cc763, State: Initialized, Role: FOLLOWER
I20260923 18:04:14.046211 1508 consensus_queue.cc:260] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14e620beab140298ae018d7de2b3769" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 37321 } } peers { permanent_uuid: "26e133bee67f4a01b82f0e12670cc763" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45011 } } peers { permanent_uuid: "1facf99c73cf45caa6b54d69c469f1e9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35541 } }
I20260923 18:04:14.046491 1509 consensus_queue.cc:260] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14e620beab140298ae018d7de2b3769" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 37321 } } peers { permanent_uuid: "26e133bee67f4a01b82f0e12670cc763" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45011 } } peers { permanent_uuid: "1facf99c73cf45caa6b54d69c469f1e9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35541 } }
I20260923 18:04:14.047260 1508 ts_tablet_manager.cc:1428] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260923 18:04:14.047451 1509 ts_tablet_manager.cc:1428] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260923 18:04:14.047705 1229 heartbeater.cc:503] Master 127.15.139.190:36023 was elected leader, sending a full tablet report...
I20260923 18:04:14.047786 1491 heartbeater.cc:503] Master 127.15.139.190:36023 was elected leader, sending a full tablet report...
I20260923 18:04:14.048933 1510 raft_consensus.cc:348] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14e620beab140298ae018d7de2b3769" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 37321 } } peers { permanent_uuid: "26e133bee67f4a01b82f0e12670cc763" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45011 } } peers { permanent_uuid: "1facf99c73cf45caa6b54d69c469f1e9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35541 } }
I20260923 18:04:14.049201 1510 raft_consensus.cc:374] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:14.049259 1510 raft_consensus.cc:729] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1facf99c73cf45caa6b54d69c469f1e9, State: Initialized, Role: FOLLOWER
I20260923 18:04:14.049659 1510 consensus_queue.cc:260] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14e620beab140298ae018d7de2b3769" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 37321 } } peers { permanent_uuid: "26e133bee67f4a01b82f0e12670cc763" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45011 } } peers { permanent_uuid: "1facf99c73cf45caa6b54d69c469f1e9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35541 } }
I20260923 18:04:14.050369 1360 heartbeater.cc:503] Master 127.15.139.190:36023 was elected leader, sending a full tablet report...
I20260923 18:04:14.050666 1510 ts_tablet_manager.cc:1428] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260923 18:04:14.075605 1514 raft_consensus.cc:482] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:04:14.075845 1514 raft_consensus.cc:504] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14e620beab140298ae018d7de2b3769" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 37321 } } peers { permanent_uuid: "26e133bee67f4a01b82f0e12670cc763" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45011 } } peers { permanent_uuid: "1facf99c73cf45caa6b54d69c469f1e9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35541 } }
I20260923 18:04:14.077127 1514 leader_election.cc:288] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26e133bee67f4a01b82f0e12670cc763 (127.15.139.131:45011), 1facf99c73cf45caa6b54d69c469f1e9 (127.15.139.130:35541)
I20260923 18:04:14.082048 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.077475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:59129 (local address 127.15.139.131:45011)
0923 18:04:14.077592 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:14.077597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:14.078016 (+ 419us) server_negotiation.cc:407] Connection header received
0923 18:04:14.078237 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:14.078240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:14.078294 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:14.078380 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:14.078720 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.079272 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.080772 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.081375 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.081506 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:14.081569 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:14.081836 (+ 267us) server_negotiation.cc:299] Negotiation successful
0923 18:04:14.081912 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 18:04:14.082048 1506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.077498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:53283 (local address 127.15.139.130:35541)
0923 18:04:14.077653 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:14.077657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:14.077671 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:04:14.077853 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:14.077855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:14.077930 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:14.078004 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:14.078416 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.079125 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.080772 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.081375 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.081506 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:14.081558 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:14.081836 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 18:04:14.081912 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 18:04:14.082072 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.077377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:35541 (local address 127.15.139.129:53283)
0923 18:04:14.077543 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:14.077591 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:14.077695 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:14.078120 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:14.078131 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:14.078155 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:14.078295 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:14.078303 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.079243 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.079249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:14.080633 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:14.080644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.081507 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.081511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:14.081679 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:14.081723 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 18:04:14.081803 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 18:04:14.081910 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":143}
I20260923 18:04:14.082089 1517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.077380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:45011 (local address 127.15.139.129:59129)
0923 18:04:14.077906 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:14.077921 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:14.078054 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:14.078421 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:14.078424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:14.078440 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:14.078588 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:14.078595 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.079350 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.079352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:14.080633 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:14.080644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.081509 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.081512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:14.081676 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:14.081722 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 18:04:14.081806 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 18:04:14.081910 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":207,"thread_start_us":105,"threads_started":1}
I20260923 18:04:14.082913 1445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28c3e81ed15a4b54b6224372af2847fc" candidate_uuid: "b14e620beab140298ae018d7de2b3769" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26e133bee67f4a01b82f0e12670cc763" is_pre_election: true
I20260923 18:04:14.082912 1314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28c3e81ed15a4b54b6224372af2847fc" candidate_uuid: "b14e620beab140298ae018d7de2b3769" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1facf99c73cf45caa6b54d69c469f1e9" is_pre_election: true
I20260923 18:04:14.083223 1314 raft_consensus.cc:2454] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b14e620beab140298ae018d7de2b3769 in term 0.
I20260923 18:04:14.083226 1445 raft_consensus.cc:2454] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b14e620beab140298ae018d7de2b3769 in term 0.
I20260923 18:04:14.083755 1117 leader_election.cc:302] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1facf99c73cf45caa6b54d69c469f1e9, 26e133bee67f4a01b82f0e12670cc763, b14e620beab140298ae018d7de2b3769; no voters:
I20260923 18:04:14.084033 1514 raft_consensus.cc:2781] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:04:14.084141 1514 raft_consensus.cc:482] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:04:14.084218 1514 raft_consensus.cc:3037] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:14.085232 1514 raft_consensus.cc:504] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14e620beab140298ae018d7de2b3769" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 37321 } } peers { permanent_uuid: "26e133bee67f4a01b82f0e12670cc763" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45011 } } peers { permanent_uuid: "1facf99c73cf45caa6b54d69c469f1e9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35541 } }
I20260923 18:04:14.085634 1514 leader_election.cc:288] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [CANDIDATE]: Term 1 election: Requested vote from peers 26e133bee67f4a01b82f0e12670cc763 (127.15.139.131:45011), 1facf99c73cf45caa6b54d69c469f1e9 (127.15.139.130:35541)
I20260923 18:04:14.086012 1314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28c3e81ed15a4b54b6224372af2847fc" candidate_uuid: "b14e620beab140298ae018d7de2b3769" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1facf99c73cf45caa6b54d69c469f1e9"
I20260923 18:04:14.086012 1445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28c3e81ed15a4b54b6224372af2847fc" candidate_uuid: "b14e620beab140298ae018d7de2b3769" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26e133bee67f4a01b82f0e12670cc763"
I20260923 18:04:14.086139 1445 raft_consensus.cc:3037] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:14.086140 1314 raft_consensus.cc:3037] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:14.087440 1445 raft_consensus.cc:2454] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b14e620beab140298ae018d7de2b3769 in term 1.
I20260923 18:04:14.087440 1314 raft_consensus.cc:2454] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b14e620beab140298ae018d7de2b3769 in term 1.
I20260923 18:04:14.087854 1116 leader_election.cc:302] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1facf99c73cf45caa6b54d69c469f1e9, b14e620beab140298ae018d7de2b3769; no voters:
I20260923 18:04:14.088074 1514 raft_consensus.cc:2781] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:14.088332 1514 raft_consensus.cc:686] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [term 1 LEADER]: Becoming Leader. State: Replica: b14e620beab140298ae018d7de2b3769, State: Running, Role: LEADER
I20260923 18:04:14.088683 1514 consensus_queue.cc:237] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14e620beab140298ae018d7de2b3769" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 37321 } } peers { permanent_uuid: "26e133bee67f4a01b82f0e12670cc763" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45011 } } peers { permanent_uuid: "1facf99c73cf45caa6b54d69c469f1e9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35541 } }
I20260923 18:04:14.092041 1039 catalog_manager.cc:5495] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 reported cstate change: term changed from 0 to 1, leader changed from <none> to b14e620beab140298ae018d7de2b3769 (127.15.139.129). New cstate: current_term: 1 leader_uuid: "b14e620beab140298ae018d7de2b3769" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14e620beab140298ae018d7de2b3769" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26e133bee67f4a01b82f0e12670cc763" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1facf99c73cf45caa6b54d69c469f1e9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35541 } health_report { overall_health: UNKNOWN } } }
W20260923 18:04:14.092914 1230 tablet.cc:2369] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:04:14.122912 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.119253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56414 (local address 127.15.139.129:37321)
0923 18:04:14.119398 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:14.119405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:14.119422 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:04:14.119546 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:14.119548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:14.119626 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:14.119750 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:14.120006 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.120506 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.121397 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:14.121633 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:14.121747 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:14.122072 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:14.122508 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:14.122660 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:14.122697 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 18:04:14.122754 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 18:04:14.129431 1445 raft_consensus.cc:1261] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763 [term 1 FOLLOWER]: Refusing update from remote peer b14e620beab140298ae018d7de2b3769: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:14.129431 1314 raft_consensus.cc:1261] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9 [term 1 FOLLOWER]: Refusing update from remote peer b14e620beab140298ae018d7de2b3769: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:14.130292 1514 consensus_queue.cc:1035] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26e133bee67f4a01b82f0e12670cc763" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 45011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:14.130529 1518 consensus_queue.cc:1035] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1facf99c73cf45caa6b54d69c469f1e9" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 35541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:14.139024 1521 mvcc.cc:205] Tried to move back new op lower bound from 7332604535305797632 to 7332604535149903872. Current Snapshot: MvccSnapshot[applied={T|T < 7332604535305797632}]
I20260923 18:04:14.139258 1522 mvcc.cc:205] Tried to move back new op lower bound from 7332604535305797632 to 7332604535149903872. Current Snapshot: MvccSnapshot[applied={T|T < 7332604535305797632}]
I20260923 18:04:14.149535 1039 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:04:14.149740 1039 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:04:14.152400 1039 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 957f7472b23241d683fe240ac536525a: Sending DeleteTablet for 3 replicas of tablet 28c3e81ed15a4b54b6224372af2847fc
I20260923 18:04:14.153301 1425 tablet_service.cc:1514] Processing DeleteTablet for tablet 28c3e81ed15a4b54b6224372af2847fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:14 UTC) from {username='slave'} at 127.0.0.1:51426
I20260923 18:04:14.153525 1294 tablet_service.cc:1514] Processing DeleteTablet for tablet 28c3e81ed15a4b54b6224372af2847fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:14 UTC) from {username='slave'} at 127.0.0.1:37110
I20260923 18:04:14.153674 1163 tablet_service.cc:1514] Processing DeleteTablet for tablet 28c3e81ed15a4b54b6224372af2847fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:14 UTC) from {username='slave'} at 127.0.0.1:56404
I20260923 18:04:14.153872 1534 tablet_replica.cc:330] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763: stopping tablet replica
I20260923 18:04:14.154086 1535 tablet_replica.cc:330] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9: stopping tablet replica
I20260923 18:04:14.154254 1534 raft_consensus.cc:2229] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:04:14.154395 1535 raft_consensus.cc:2229] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:04:14.154659 1534 raft_consensus.cc:2258] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:14.154692 1535 raft_consensus.cc:2258] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:14.155128 1536 tablet_replica.cc:330] T 28c3e81ed15a4b54b6224372af2847fc P b14e620beab140298ae018d7de2b3769: stopping tablet replica
I20260923 18:04:14.155418 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 1101
I20260923 18:04:14.157186 1534 ts_tablet_manager.cc:1905] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:04:14.157974 1535 ts_tablet_manager.cc:1905] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:04:14.160286 1535 ts_tablet_manager.cc:1918] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:04:14.160404 1535 log.cc:1196] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/28c3e81ed15a4b54b6224372af2847fc
I20260923 18:04:14.160727 1535 ts_tablet_manager.cc:1939] T 28c3e81ed15a4b54b6224372af2847fc P 1facf99c73cf45caa6b54d69c469f1e9: Deleting consensus metadata
I20260923 18:04:14.160763 1534 ts_tablet_manager.cc:1918] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:04:14.160862 1534 log.cc:1196] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/28c3e81ed15a4b54b6224372af2847fc
I20260923 18:04:14.161180 1534 ts_tablet_manager.cc:1939] T 28c3e81ed15a4b54b6224372af2847fc P 26e133bee67f4a01b82f0e12670cc763: Deleting consensus metadata
I20260923 18:04:14.162042 1025 catalog_manager.cc:4841] TS 1facf99c73cf45caa6b54d69c469f1e9 (127.15.139.130:35541): tablet 28c3e81ed15a4b54b6224372af2847fc (table test-table [id=2b1ecae678be4b22bf7dea63de8101e6]) successfully deleted
I20260923 18:04:14.162366 1026 catalog_manager.cc:4841] TS 26e133bee67f4a01b82f0e12670cc763 (127.15.139.131:45011): tablet 28c3e81ed15a4b54b6224372af2847fc (table test-table [id=2b1ecae678be4b22bf7dea63de8101e6]) successfully deleted
W20260923 18:04:14.163260 1028 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:37321 (error 108)
I20260923 18:04:14.163412 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 1232
I20260923 18:04:14.165205 1502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:14.164819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:37321 (local address 127.0.0.1:56418)
0923 18:04:14.164988 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:14.165074 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:37321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260923 18:04:14.165377 1028 catalog_manager.cc:4584] TS b14e620beab140298ae018d7de2b3769 (127.15.139.129:37321): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28c3e81ed15a4b54b6224372af2847fc: Network error: Client connection negotiation failed: client connection to 127.15.139.129:37321: connect: Connection refused (error 111)
I20260923 18:04:14.169714 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 1363
I20260923 18:04:14.175320 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 1010
2026-09-23T18:04:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1049 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:14 dist-test-slave-vr57 krb5kdc[1540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:14 dist-test-slave-vr57 krb5kdc[1540](info): set up 2 sockets
Sep 23 18:04:14 dist-test-slave-vr57 krb5kdc[1540](info): commencing operation
krb5kdc: starting...
W20260923 18:04:15.235956 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:15 dist-test-slave-vr57 krb5kdc[1540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:04:15.363144 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36589
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:36589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:15.476508 1553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:15.476799 1553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:15.476890 1553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:15.481103 1553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:15.481168 1553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:15.481189 1553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:15.481205 1553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:15.481220 1553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:15.486743 1553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:36589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:15.487874 1553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:15.488919 1553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:15.497430 1558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:15.497450 1559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:15.497457 1561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:15.497725 1553 server_base.cc:1034] running on GCE node
I20260923 18:04:15.498383 1553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:15.499468 1553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:15.500705 1553 hybrid_clock.cc:648] HybridClock initialized: now 1790186655500683 us; error 34 us; skew 500 ppm
Sep 23 18:04:15 dist-test-slave-vr57 krb5kdc[1540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186655, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:15.503558 1553 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:04:15.504459 1553 webserver.cc:460] Webserver started at http://127.15.139.190:34141/ using document root <none> and password file <none>
I20260923 18:04:15.504964 1553 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:15.505021 1553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:15.505185 1553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:15.507175 1553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "7df906d1aed343338835849850f74170"
format_stamp: "Formatted at 2026-09-23 18:04:15 on dist-test-slave-vr57"
server_key: "87ac205f3141f7fc452fe665b6a9bc20"
server_key_iv: "d49f7a155b3345e6f0473aba928cd148"
server_key_version: "encryptionkey@0"
I20260923 18:04:15.507761 1553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "7df906d1aed343338835849850f74170"
format_stamp: "Formatted at 2026-09-23 18:04:15 on dist-test-slave-vr57"
server_key: "87ac205f3141f7fc452fe665b6a9bc20"
server_key_iv: "d49f7a155b3345e6f0473aba928cd148"
server_key_version: "encryptionkey@0"
I20260923 18:04:15.511475 1553 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:04:15.514045 1568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:15.515393 1553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:04:15.515630 1553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "7df906d1aed343338835849850f74170"
format_stamp: "Formatted at 2026-09-23 18:04:15 on dist-test-slave-vr57"
server_key: "87ac205f3141f7fc452fe665b6a9bc20"
server_key_iv: "d49f7a155b3345e6f0473aba928cd148"
server_key_version: "encryptionkey@0"
I20260923 18:04:15.515748 1553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:15.533221 1553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:15.536070 1553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:15.536230 1553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:15.543673 1553 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:36589
I20260923 18:04:15.543704 1620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:36589 every 8 connection(s)
I20260923 18:04:15.544689 1553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:15.547421 1621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:15.549654 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 1553
I20260923 18:04:15.549803 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:15.550107 15918 external_mini_cluster.cc:1428] Setting key ad860a751b6bddd66f05cc4f9c83960a
I20260923 18:04:15.553789 1621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170: Bootstrap starting.
I20260923 18:04:15.556078 1621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:15.556814 1621 log.cc:824] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:15.558820 1621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170: No bootstrap required, opened a new log
Sep 23 18:04:15 dist-test-slave-vr57 krb5kdc[1540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:15.562233 1621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df906d1aed343338835849850f74170" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36589 } }
I20260923 18:04:15.562484 1621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:15.562569 1621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df906d1aed343338835849850f74170, State: Initialized, Role: FOLLOWER
I20260923 18:04:15.563133 1621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df906d1aed343338835849850f74170" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36589 } }
I20260923 18:04:15.563310 1621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:15.563382 1621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:15.563529 1621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:15.564642 1621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df906d1aed343338835849850f74170" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36589 } }
I20260923 18:04:15.565028 1621 leader_election.cc:302] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7df906d1aed343338835849850f74170; no voters:
I20260923 18:04:15.565363 1621 leader_election.cc:288] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:15.565475 1626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:15.565726 1626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [term 1 LEADER]: Becoming Leader. State: Replica: 7df906d1aed343338835849850f74170, State: Running, Role: LEADER
I20260923 18:04:15.565833 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:15.551610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39794 (local address 127.15.139.190:36589)
0923 18:04:15.552094 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:15.552109 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:15.552145 (+ 36us) server_negotiation.cc:407] Connection header received
0923 18:04:15.552997 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:15.553016 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:15.553354 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:15.553754 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:15.554642 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:15.555717 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:15.556685 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:15.557215 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:15.557394 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:15.560038 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:15.560042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:15.560056 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:15.560084 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:15.562515 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:15.563039 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:15.563044 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:15.563048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:15.563162 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:15.563484 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:15.563488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:15.563489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:15.564009 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:15.564166 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:15.564470 (+ 304us) server_negotiation.cc:299] Negotiation successful
0923 18:04:15.564815 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":141,"threads_started":1}
I20260923 18:04:15.566120 1626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df906d1aed343338835849850f74170" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36589 } }
I20260923 18:04:15.566629 1621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:15.568729 1628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7df906d1aed343338835849850f74170. Latest consensus state: current_term: 1 leader_uuid: "7df906d1aed343338835849850f74170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df906d1aed343338835849850f74170" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36589 } } }
I20260923 18:04:15.569155 1628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:15.568935 1627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7df906d1aed343338835849850f74170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df906d1aed343338835849850f74170" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36589 } } }
I20260923 18:04:15.569391 1627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170 [sys.catalog]: This master's current role is: LEADER
W20260923 18:04:15.571647 1640 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:04:15.571717 1640 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:04:15.571846 1642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:15.572652 1642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:15.578542 1642 catalog_manager.cc:1347] Generated new cluster ID: ba286b66d0624bb680b5864242ad813c
I20260923 18:04:15.578615 1642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:15.610513 1642 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:15.611881 1642 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:15.630090 1642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170: Generated new TSK 0
I20260923 18:04:15.631053 1642 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:04:15.662406 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36589
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:15.776388 1647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:15.776670 1647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:15.776721 1647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:15.780977 1647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:15.781033 1647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:15.781091 1647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:04:15.786484 1647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:15.787591 1647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:15.788686 1647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:15.798532 1655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:15.798641 1652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:15.798707 1647 server_base.cc:1034] running on GCE node
W20260923 18:04:15.798566 1653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:15.799295 1647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:15.799978 1647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:15.801165 1647 hybrid_clock.cc:648] HybridClock initialized: now 1790186655801131 us; error 50 us; skew 500 ppm
Sep 23 18:04:15 dist-test-slave-vr57 krb5kdc[1540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186655, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:15.804277 1647 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:04:15.805284 1647 webserver.cc:460] Webserver started at http://127.15.139.129:42877/ using document root <none> and password file <none>
I20260923 18:04:15.805881 1647 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:15.805990 1647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:15.806243 1647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:15.808497 1647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "0be93de7aa664621b6cf41d3126e47f5"
format_stamp: "Formatted at 2026-09-23 18:04:15 on dist-test-slave-vr57"
server_key: "cd5553c44fce10863ea39283bf94b084"
server_key_iv: "e74e8da7eff0132f38f6717ebbb4e300"
server_key_version: "encryptionkey@0"
I20260923 18:04:15.809139 1647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "0be93de7aa664621b6cf41d3126e47f5"
format_stamp: "Formatted at 2026-09-23 18:04:15 on dist-test-slave-vr57"
server_key: "cd5553c44fce10863ea39283bf94b084"
server_key_iv: "e74e8da7eff0132f38f6717ebbb4e300"
server_key_version: "encryptionkey@0"
I20260923 18:04:15.813568 1647 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:04:15.816344 1662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:15.817672 1647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:15.817844 1647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "0be93de7aa664621b6cf41d3126e47f5"
format_stamp: "Formatted at 2026-09-23 18:04:15 on dist-test-slave-vr57"
server_key: "cd5553c44fce10863ea39283bf94b084"
server_key_iv: "e74e8da7eff0132f38f6717ebbb4e300"
server_key_version: "encryptionkey@0"
I20260923 18:04:15.817978 1647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:15.831168 1647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:15.834316 1647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:15.834553 1647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:15.835181 1647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:15.836153 1647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:15.836226 1647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:15.836306 1647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:15.836364 1647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:15.845582 1647 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:44163
I20260923 18:04:15.845631 1775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:44163 every 8 connection(s)
I20260923 18:04:15.846601 1647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:04:15.849186 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 1647
I20260923 18:04:15.849346 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:04:15.849793 15918 external_mini_cluster.cc:1428] Setting key e77f79ee65e43aac1489b8a995be9aae
Sep 23 18:04:15 dist-test-slave-vr57 krb5kdc[1540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186655, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
I20260923 18:04:15.862881 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:15.848537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:42969 (local address 127.15.139.190:36589)
0923 18:04:15.848709 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:15.848714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:15.849618 (+ 904us) server_negotiation.cc:407] Connection header received
0923 18:04:15.851226 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:15.851229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:15.851287 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:15.851392 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:15.853118 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:15.854047 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:15.855310 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:15.855807 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:15.855951 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:15.858573 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:15.858579 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:15.858584 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:15.858620 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:15.860503 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:15.861129 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:15.861132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:15.861133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:15.861219 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:15.861666 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:15.861672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:15.861677 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:15.861897 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:15.861991 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:15.862641 (+ 650us) server_negotiation.cc:299] Negotiation successful
0923 18:04:15.862755 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
I20260923 18:04:15.864084 1778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:15.848951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36589 (local address 127.15.139.129:42969)
0923 18:04:15.849435 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:15.849490 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:15.850950 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:15.851640 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:15.851650 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:15.852193 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:15.852912 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:15.852945 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:15.854224 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:15.854226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:15.855166 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:15.855175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:15.856041 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:15.856048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:15.856216 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:15.856840 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:15.856857 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:15.858371 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:15.860655 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:15.860664 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:15.860688 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:15.861016 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:15.861312 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:15.861315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:15.861316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:15.861516 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:15.862045 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:15.862051 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:15.862314 (+ 263us) client_negotiation.cc:769] Sending connection context
0923 18:04:15.862568 (+ 254us) client_negotiation.cc:240] Negotiation successful
0923 18:04:15.862896 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":134,"threads_started":1}
I20260923 18:04:15.865567 1776 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36589
I20260923 18:04:15.865865 1776 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:15.866569 1776 heartbeater.cc:511] Master 127.15.139.190:36589 requested a full tablet report, sending...
I20260923 18:04:15.868285 1585 ts_manager.cc:194] Registered new tserver with Master: 0be93de7aa664621b6cf41d3126e47f5 (127.15.139.129:44163)
I20260923 18:04:15.869578 1585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:42969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:04:15.882597 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36589
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:16.005386 1781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:16.005725 1781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:16.005779 1781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:16.010582 1781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:16.010659 1781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:16.010730 1781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:04:16.016683 1781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:16.017807 1781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:16.018904 1781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:16.028963 1787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:16.028972 1789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:16.029049 1781 server_base.cc:1034] running on GCE node
W20260923 18:04:16.028963 1786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:16.029794 1781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:16.030447 1781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:16.031589 1781 hybrid_clock.cc:648] HybridClock initialized: now 1790186656031559 us; error 48 us; skew 500 ppm
Sep 23 18:04:16 dist-test-slave-vr57 krb5kdc[1540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186656, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:16.034361 1781 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:04:16.035247 1781 webserver.cc:460] Webserver started at http://127.15.139.130:46723/ using document root <none> and password file <none>
I20260923 18:04:16.035851 1781 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:16.035917 1781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:16.036090 1781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:16.038102 1781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "5966a18ccdbc449eacc02573cdafd38e"
format_stamp: "Formatted at 2026-09-23 18:04:16 on dist-test-slave-vr57"
server_key: "c7a134e4cd75ec2623cf538a5a817338"
server_key_iv: "bee2a50a60fe40b50beb0f7989eaba28"
server_key_version: "encryptionkey@0"
I20260923 18:04:16.038668 1781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "5966a18ccdbc449eacc02573cdafd38e"
format_stamp: "Formatted at 2026-09-23 18:04:16 on dist-test-slave-vr57"
server_key: "c7a134e4cd75ec2623cf538a5a817338"
server_key_iv: "bee2a50a60fe40b50beb0f7989eaba28"
server_key_version: "encryptionkey@0"
I20260923 18:04:16.042526 1781 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260923 18:04:16.045087 1796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:16.046579 1781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260923 18:04:16.046725 1781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "5966a18ccdbc449eacc02573cdafd38e"
format_stamp: "Formatted at 2026-09-23 18:04:16 on dist-test-slave-vr57"
server_key: "c7a134e4cd75ec2623cf538a5a817338"
server_key_iv: "bee2a50a60fe40b50beb0f7989eaba28"
server_key_version: "encryptionkey@0"
I20260923 18:04:16.046839 1781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:16.060793 1781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:16.064262 1781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:16.064491 1781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:16.065121 1781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:16.066000 1781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:16.066074 1781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:16.066147 1781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:16.066213 1781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:16.075323 1781 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:45139
I20260923 18:04:16.075349 1909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:45139 every 8 connection(s)
I20260923 18:04:16.076484 1781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:04:16.080237 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 1781
I20260923 18:04:16.080406 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:04:16.080711 15918 external_mini_cluster.cc:1428] Setting key ed8b1ecee75fc60c09e579a070ab5912
Sep 23 18:04:16 dist-test-slave-vr57 krb5kdc[1540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186656, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
I20260923 18:04:16.093060 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.078618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:33927 (local address 127.15.139.190:36589)
0923 18:04:16.078799 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:16.078804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:16.079833 (+ 1029us) server_negotiation.cc:407] Connection header received
0923 18:04:16.081418 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:16.081421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:16.081498 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:16.081593 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:16.082947 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.083839 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.085775 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.085994 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.086081 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.089028 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:16.089032 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:16.089035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:16.089070 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:16.090868 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:16.091435 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:16.091442 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:16.091446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:16.091583 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:16.091958 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:16.091961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:16.091962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:16.092152 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:16.092233 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:16.092820 (+ 587us) server_negotiation.cc:299] Negotiation successful
0923 18:04:16.092933 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
I20260923 18:04:16.094125 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.079055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36589 (local address 127.15.139.130:33927)
0923 18:04:16.079677 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:16.079724 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:16.081157 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:16.081781 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:16.081789 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:16.082273 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:16.082790 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.082804 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.083998 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.084004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.085624 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.085640 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.086167 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.086173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.086428 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.087162 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:16.087185 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:16.088841 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:16.091001 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:16.091007 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:16.091021 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:16.091321 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:16.091704 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:16.091707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:16.091708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:16.091860 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:16.092247 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:16.092252 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:16.092591 (+ 339us) client_negotiation.cc:769] Sending connection context
0923 18:04:16.092761 (+ 170us) client_negotiation.cc:240] Negotiation successful
0923 18:04:16.093035 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":119,"threads_started":1}
I20260923 18:04:16.095620 1910 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36589
I20260923 18:04:16.095911 1910 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:16.096510 1910 heartbeater.cc:511] Master 127.15.139.190:36589 requested a full tablet report, sending...
I20260923 18:04:16.097653 1585 ts_manager.cc:194] Registered new tserver with Master: 5966a18ccdbc449eacc02573cdafd38e (127.15.139.130:45139)
I20260923 18:04:16.098290 1585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:33927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:04:16.113376 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36589
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:16.227330 1915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:16.227670 1915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:16.227720 1915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:16.231731 1915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:16.231791 1915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:16.231854 1915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:04:16.237341 1915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:16.238466 1915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:16.239475 1915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:16.249434 1923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:16.249454 1921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:16.249650 1915 server_base.cc:1034] running on GCE node
W20260923 18:04:16.249434 1920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:16.250249 1915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:16.250893 1915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:16.252132 1915 hybrid_clock.cc:648] HybridClock initialized: now 1790186656252118 us; error 28 us; skew 500 ppm
Sep 23 18:04:16 dist-test-slave-vr57 krb5kdc[1540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186656, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:16.255144 1915 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:04:16.256188 1915 webserver.cc:460] Webserver started at http://127.15.139.131:33527/ using document root <none> and password file <none>
I20260923 18:04:16.256860 1915 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:16.256955 1915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:16.257195 1915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:16.259285 1915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "ac5ccc8b5c81446a9797e2445c6787ef"
format_stamp: "Formatted at 2026-09-23 18:04:16 on dist-test-slave-vr57"
server_key: "90c9adbb6025c21825daacd87b2fac6c"
server_key_iv: "0fcc4c83dec0d8b42f883c08d0a063c4"
server_key_version: "encryptionkey@0"
I20260923 18:04:16.259953 1915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "ac5ccc8b5c81446a9797e2445c6787ef"
format_stamp: "Formatted at 2026-09-23 18:04:16 on dist-test-slave-vr57"
server_key: "90c9adbb6025c21825daacd87b2fac6c"
server_key_iv: "0fcc4c83dec0d8b42f883c08d0a063c4"
server_key_version: "encryptionkey@0"
I20260923 18:04:16.263841 1915 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:04:16.266526 1930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:16.268124 1915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:16.268276 1915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "ac5ccc8b5c81446a9797e2445c6787ef"
format_stamp: "Formatted at 2026-09-23 18:04:16 on dist-test-slave-vr57"
server_key: "90c9adbb6025c21825daacd87b2fac6c"
server_key_iv: "0fcc4c83dec0d8b42f883c08d0a063c4"
server_key_version: "encryptionkey@0"
I20260923 18:04:16.268401 1915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:16.280441 1915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:16.283368 1915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:16.283622 1915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:16.284262 1915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:16.285158 1915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:16.285233 1915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:16.285302 1915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:16.285404 1915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260923 18:04:16.294570 1915 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:43107
I20260923 18:04:16.294592 2043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:43107 every 8 connection(s)
I20260923 18:04:16.295552 1915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:04:16.300522 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 1915
I20260923 18:04:16.300689 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:04:16.301002 15918 external_mini_cluster.cc:1428] Setting key bae387914a0fe8320ff086f251058646
Sep 23 18:04:16 dist-test-slave-vr57 krb5kdc[1540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186656, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:16.310186 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.297543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:40041 (local address 127.15.139.190:36589)
0923 18:04:16.297710 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:16.297721 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:16.298510 (+ 789us) server_negotiation.cc:407] Connection header received
0923 18:04:16.299951 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:16.299954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:16.300001 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:16.300080 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:16.301468 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.302086 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.303670 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.303885 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.303966 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.306515 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:16.306518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:16.306520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:16.306545 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:16.308149 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:16.308755 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:16.308758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:16.308759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:16.308837 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:16.309204 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:16.309206 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:16.309207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:16.309381 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:16.309455 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:16.309954 (+ 499us) server_negotiation.cc:299] Negotiation successful
0923 18:04:16.310053 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260923 18:04:16.311198 2046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.297868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36589 (local address 127.15.139.131:40041)
0923 18:04:16.298347 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:16.298390 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:16.299746 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:16.300243 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:16.300251 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:16.300702 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:16.301318 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.301332 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.302229 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.302232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.303559 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.303569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.304042 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.304045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.304190 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.304756 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:16.304772 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:16.306336 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:16.308275 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:16.308281 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:16.308294 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:16.308643 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:16.308958 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:16.308961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:16.308962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:16.309118 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:16.309477 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:16.309483 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:16.309744 (+ 261us) client_negotiation.cc:769] Sending connection context
0923 18:04:16.309945 (+ 201us) client_negotiation.cc:240] Negotiation successful
0923 18:04:16.310273 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":122,"threads_started":1}
I20260923 18:04:16.312551 2044 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36589
I20260923 18:04:16.312842 2044 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:16.313406 2044 heartbeater.cc:511] Master 127.15.139.190:36589 requested a full tablet report, sending...
I20260923 18:04:16.314738 1585 ts_manager.cc:194] Registered new tserver with Master: ac5ccc8b5c81446a9797e2445c6787ef (127.15.139.131:43107)
I20260923 18:04:16.315267 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:04:16.315562 1585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:40041
I20260923 18:04:16.325635 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.317518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50164 (local address 127.15.139.190:36589)
0923 18:04:16.317663 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:16.317668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:16.317809 (+ 141us) server_negotiation.cc:407] Connection header received
0923 18:04:16.317927 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:16.317929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:16.317963 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:16.318027 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:16.318671 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.319223 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.320211 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.320431 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.320512 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.322266 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:16.322269 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:16.322271 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:16.322294 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:16.323909 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:16.324451 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:16.324453 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:16.324454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:16.324532 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:16.324817 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:16.324819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:16.324820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:16.325020 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:16.325159 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:16.325355 (+ 196us) server_negotiation.cc:299] Negotiation successful
0923 18:04:16.325434 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 18:04:16.329110 1585 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:04:16.332291 1585 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:04:16.347409 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.341155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41276 (local address 127.15.139.129:44163)
0923 18:04:16.341558 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:16.341577 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:16.342113 (+ 536us) server_negotiation.cc:407] Connection header received
0923 18:04:16.342426 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:16.342432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:16.342665 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:16.342796 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:16.343429 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.344603 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.346051 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.346654 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.346795 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.346937 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:16.347112 (+ 175us) server_negotiation.cc:299] Negotiation successful
0923 18:04:16.347249 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":148,"threads_started":1}
I20260923 18:04:16.347565 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.341715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55040 (local address 127.15.139.130:45139)
0923 18:04:16.342543 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:16.342555 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:16.342575 (+ 20us) server_negotiation.cc:407] Connection header received
0923 18:04:16.342644 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:16.342649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:16.342803 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:16.342946 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:16.343385 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.344604 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.346081 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.346886 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.347036 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.347157 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:16.347265 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 18:04:16.347375 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":104,"threads_started":1}
I20260923 18:04:16.347584 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.341296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:45139 (local address 127.0.0.1:55040)
0923 18:04:16.342093 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:16.342106 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:16.342241 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:16.342972 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:16.342979 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:16.343007 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:16.343257 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.343266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.344753 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.344757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.345934 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.345942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.347029 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.347033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.347126 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.347163 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 18:04:16.347234 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 18:04:16.347358 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":100,"spinlock_wait_cycles":5632,"thread_start_us":116,"threads_started":1}
I20260923 18:04:16.347594 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.340974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:43107 (local address 127.0.0.1:39690)
0923 18:04:16.342142 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:16.342157 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:16.342322 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:16.343046 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:16.343048 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:16.343060 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:16.343279 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.343285 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.344358 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.344365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.345709 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.345719 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.347109 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.347113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.347255 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.347280 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 18:04:16.347352 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 18:04:16.347417 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1096,"mutex_wait_us":992,"thread_start_us":133,"threads_started":1}
I20260923 18:04:16.348047 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.341555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39690 (local address 127.15.139.131:43107)
0923 18:04:16.341888 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:16.341901 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:16.342247 (+ 346us) server_negotiation.cc:407] Connection header received
0923 18:04:16.342490 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:16.342496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:16.342707 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:16.342839 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:16.343411 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.344228 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.345839 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.346768 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.346896 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.347046 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:16.347662 (+ 616us) server_negotiation.cc:299] Negotiation successful
0923 18:04:16.347782 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":159,"threads_started":1}
I20260923 18:04:16.348301 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.340967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:44163 (local address 127.0.0.1:41276)
0923 18:04:16.341964 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:16.342019 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:16.342241 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:16.342996 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:16.342999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:16.343016 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:16.343273 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.343280 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.344786 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.344788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.345917 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.345929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.346780 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.346784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.346887 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.346924 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 18:04:16.348082 (+ 1158us) client_negotiation.cc:240] Negotiation successful
0923 18:04:16.348175 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":131,"spinlock_wait_cycles":7168,"thread_start_us":86,"threads_started":1}
I20260923 18:04:16.350800 1710 tablet_service.cc:1467] Processing CreateTablet for tablet c93dd1663ef548db945a05b16b321ec5 (DEFAULT_TABLE table=test-table [id=28374436c3304c34849db32b0a321d99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:16.350807 1978 tablet_service.cc:1467] Processing CreateTablet for tablet c93dd1663ef548db945a05b16b321ec5 (DEFAULT_TABLE table=test-table [id=28374436c3304c34849db32b0a321d99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:16.350807 1844 tablet_service.cc:1467] Processing CreateTablet for tablet c93dd1663ef548db945a05b16b321ec5 (DEFAULT_TABLE table=test-table [id=28374436c3304c34849db32b0a321d99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:16.351900 1710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c93dd1663ef548db945a05b16b321ec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:16.351902 1978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c93dd1663ef548db945a05b16b321ec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:16.351902 1844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c93dd1663ef548db945a05b16b321ec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:16.358227 2061 tablet_bootstrap.cc:492] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5: Bootstrap starting.
I20260923 18:04:16.359874 2063 tablet_bootstrap.cc:492] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e: Bootstrap starting.
I20260923 18:04:16.359992 2062 tablet_bootstrap.cc:492] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef: Bootstrap starting.
I20260923 18:04:16.360781 2061 tablet_bootstrap.cc:654] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:16.362013 2062 tablet_bootstrap.cc:654] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:16.362243 2063 tablet_bootstrap.cc:654] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:16.362241 2061 log.cc:824] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:16.363314 2062 log.cc:824] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:16.363483 2063 log.cc:824] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:16.364631 2061 tablet_bootstrap.cc:492] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5: No bootstrap required, opened a new log
I20260923 18:04:16.364931 2061 ts_tablet_manager.cc:1397] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260923 18:04:16.365468 2062 tablet_bootstrap.cc:492] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef: No bootstrap required, opened a new log
I20260923 18:04:16.365547 2063 tablet_bootstrap.cc:492] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e: No bootstrap required, opened a new log
I20260923 18:04:16.365649 2062 ts_tablet_manager.cc:1397] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260923 18:04:16.365804 2063 ts_tablet_manager.cc:1397] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260923 18:04:16.368991 2063 raft_consensus.cc:348] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be93de7aa664621b6cf41d3126e47f5" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44163 } } peers { permanent_uuid: "ac5ccc8b5c81446a9797e2445c6787ef" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43107 } } peers { permanent_uuid: "5966a18ccdbc449eacc02573cdafd38e" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45139 } }
I20260923 18:04:16.369220 2063 raft_consensus.cc:374] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:16.369256 2063 raft_consensus.cc:729] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5966a18ccdbc449eacc02573cdafd38e, State: Initialized, Role: FOLLOWER
I20260923 18:04:16.369243 2061 raft_consensus.cc:348] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be93de7aa664621b6cf41d3126e47f5" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44163 } } peers { permanent_uuid: "ac5ccc8b5c81446a9797e2445c6787ef" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43107 } } peers { permanent_uuid: "5966a18ccdbc449eacc02573cdafd38e" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45139 } }
I20260923 18:04:16.369495 2061 raft_consensus.cc:374] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:16.369542 2061 raft_consensus.cc:729] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be93de7aa664621b6cf41d3126e47f5, State: Initialized, Role: FOLLOWER
I20260923 18:04:16.369673 2063 consensus_queue.cc:260] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be93de7aa664621b6cf41d3126e47f5" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44163 } } peers { permanent_uuid: "ac5ccc8b5c81446a9797e2445c6787ef" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43107 } } peers { permanent_uuid: "5966a18ccdbc449eacc02573cdafd38e" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45139 } }
I20260923 18:04:16.369575 2062 raft_consensus.cc:348] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be93de7aa664621b6cf41d3126e47f5" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44163 } } peers { permanent_uuid: "ac5ccc8b5c81446a9797e2445c6787ef" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43107 } } peers { permanent_uuid: "5966a18ccdbc449eacc02573cdafd38e" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45139 } }
I20260923 18:04:16.369792 2062 raft_consensus.cc:374] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:16.369838 2062 raft_consensus.cc:729] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac5ccc8b5c81446a9797e2445c6787ef, State: Initialized, Role: FOLLOWER
I20260923 18:04:16.370013 2061 consensus_queue.cc:260] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be93de7aa664621b6cf41d3126e47f5" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44163 } } peers { permanent_uuid: "ac5ccc8b5c81446a9797e2445c6787ef" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43107 } } peers { permanent_uuid: "5966a18ccdbc449eacc02573cdafd38e" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45139 } }
I20260923 18:04:16.370198 2062 consensus_queue.cc:260] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be93de7aa664621b6cf41d3126e47f5" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44163 } } peers { permanent_uuid: "ac5ccc8b5c81446a9797e2445c6787ef" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43107 } } peers { permanent_uuid: "5966a18ccdbc449eacc02573cdafd38e" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45139 } }
I20260923 18:04:16.370361 1910 heartbeater.cc:503] Master 127.15.139.190:36589 was elected leader, sending a full tablet report...
I20260923 18:04:16.370646 1776 heartbeater.cc:503] Master 127.15.139.190:36589 was elected leader, sending a full tablet report...
I20260923 18:04:16.371034 2061 ts_tablet_manager.cc:1428] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260923 18:04:16.371034 2062 ts_tablet_manager.cc:1428] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260923 18:04:16.371649 2044 heartbeater.cc:503] Master 127.15.139.190:36589 was elected leader, sending a full tablet report...
I20260923 18:04:16.372000 2063 ts_tablet_manager.cc:1428] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260923 18:04:16.381769 2068 raft_consensus.cc:482] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:04:16.381974 2068 raft_consensus.cc:504] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be93de7aa664621b6cf41d3126e47f5" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44163 } } peers { permanent_uuid: "ac5ccc8b5c81446a9797e2445c6787ef" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43107 } } peers { permanent_uuid: "5966a18ccdbc449eacc02573cdafd38e" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45139 } }
I20260923 18:04:16.383075 2068 leader_election.cc:288] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac5ccc8b5c81446a9797e2445c6787ef (127.15.139.131:43107), 5966a18ccdbc449eacc02573cdafd38e (127.15.139.130:45139)
I20260923 18:04:16.388550 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.383425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:40543 (local address 127.15.139.130:45139)
0923 18:04:16.383610 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:16.383616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:16.383885 (+ 269us) server_negotiation.cc:407] Connection header received
0923 18:04:16.384030 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:16.384033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:16.384084 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:16.384222 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:16.384599 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.385383 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.387238 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.387887 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.388053 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.388096 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:16.388326 (+ 230us) server_negotiation.cc:299] Negotiation successful
0923 18:04:16.388410 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260923 18:04:16.388574 2071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.383323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:45139 (local address 127.15.139.129:40543)
0923 18:04:16.383789 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:16.383804 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:16.383926 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:16.384224 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:16.384226 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:16.384238 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:16.384452 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.384459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.385548 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.385553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.387065 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.387072 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.388047 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.388049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.388162 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.388189 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 18:04:16.388276 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 18:04:16.388357 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":157,"thread_start_us":122,"threads_started":1}
I20260923 18:04:16.388713 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.383415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:41899 (local address 127.15.139.131:43107)
0923 18:04:16.383601 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:16.383608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:16.383932 (+ 324us) server_negotiation.cc:407] Connection header received
0923 18:04:16.384030 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:16.384033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:16.384084 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:16.384173 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:16.384598 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.385383 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.387238 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.387895 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.388033 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.388087 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:16.388496 (+ 409us) server_negotiation.cc:299] Negotiation successful
0923 18:04:16.388588 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260923 18:04:16.389071 2070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.383323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:43107 (local address 127.15.139.129:41899)
0923 18:04:16.383840 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:16.383853 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:16.383928 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:16.384204 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:16.384209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:16.384228 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:16.384452 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.384459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.385657 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.385660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.387059 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.387072 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.388034 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.388038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.388355 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.388374 (+ 19us) client_negotiation.cc:769] Sending connection context
0923 18:04:16.388844 (+ 470us) client_negotiation.cc:240] Negotiation successful
0923 18:04:16.388922 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":149,"thread_start_us":128,"threads_started":1}
I20260923 18:04:16.389308 1864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93dd1663ef548db945a05b16b321ec5" candidate_uuid: "0be93de7aa664621b6cf41d3126e47f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5966a18ccdbc449eacc02573cdafd38e" is_pre_election: true
I20260923 18:04:16.389670 1864 raft_consensus.cc:2454] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0be93de7aa664621b6cf41d3126e47f5 in term 0.
I20260923 18:04:16.389700 1998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93dd1663ef548db945a05b16b321ec5" candidate_uuid: "0be93de7aa664621b6cf41d3126e47f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac5ccc8b5c81446a9797e2445c6787ef" is_pre_election: true
I20260923 18:04:16.390022 1998 raft_consensus.cc:2454] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0be93de7aa664621b6cf41d3126e47f5 in term 0.
I20260923 18:04:16.390200 1663 leader_election.cc:302] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0be93de7aa664621b6cf41d3126e47f5, 5966a18ccdbc449eacc02573cdafd38e; no voters:
I20260923 18:04:16.390494 2068 raft_consensus.cc:2781] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:04:16.390573 2068 raft_consensus.cc:482] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:04:16.390604 2068 raft_consensus.cc:3037] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:16.391784 2068 raft_consensus.cc:504] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be93de7aa664621b6cf41d3126e47f5" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44163 } } peers { permanent_uuid: "ac5ccc8b5c81446a9797e2445c6787ef" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43107 } } peers { permanent_uuid: "5966a18ccdbc449eacc02573cdafd38e" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45139 } }
I20260923 18:04:16.392159 2068 leader_election.cc:288] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [CANDIDATE]: Term 1 election: Requested vote from peers ac5ccc8b5c81446a9797e2445c6787ef (127.15.139.131:43107), 5966a18ccdbc449eacc02573cdafd38e (127.15.139.130:45139)
I20260923 18:04:16.392530 1998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93dd1663ef548db945a05b16b321ec5" candidate_uuid: "0be93de7aa664621b6cf41d3126e47f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac5ccc8b5c81446a9797e2445c6787ef"
I20260923 18:04:16.392530 1864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93dd1663ef548db945a05b16b321ec5" candidate_uuid: "0be93de7aa664621b6cf41d3126e47f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5966a18ccdbc449eacc02573cdafd38e"
I20260923 18:04:16.392679 1998 raft_consensus.cc:3037] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:16.392679 1864 raft_consensus.cc:3037] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:16.393985 1998 raft_consensus.cc:2454] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0be93de7aa664621b6cf41d3126e47f5 in term 1.
I20260923 18:04:16.393982 1864 raft_consensus.cc:2454] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0be93de7aa664621b6cf41d3126e47f5 in term 1.
I20260923 18:04:16.394312 1663 leader_election.cc:302] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0be93de7aa664621b6cf41d3126e47f5, 5966a18ccdbc449eacc02573cdafd38e, ac5ccc8b5c81446a9797e2445c6787ef; no voters:
I20260923 18:04:16.394460 2068 raft_consensus.cc:2781] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:16.394687 2068 raft_consensus.cc:686] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 1 LEADER]: Becoming Leader. State: Replica: 0be93de7aa664621b6cf41d3126e47f5, State: Running, Role: LEADER
I20260923 18:04:16.395020 2068 consensus_queue.cc:237] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be93de7aa664621b6cf41d3126e47f5" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44163 } } peers { permanent_uuid: "ac5ccc8b5c81446a9797e2445c6787ef" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43107 } } peers { permanent_uuid: "5966a18ccdbc449eacc02573cdafd38e" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45139 } }
I20260923 18:04:16.398372 1585 catalog_manager.cc:5495] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0be93de7aa664621b6cf41d3126e47f5 (127.15.139.129). New cstate: current_term: 1 leader_uuid: "0be93de7aa664621b6cf41d3126e47f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be93de7aa664621b6cf41d3126e47f5" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 44163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac5ccc8b5c81446a9797e2445c6787ef" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5966a18ccdbc449eacc02573cdafd38e" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45139 } health_report { overall_health: UNKNOWN } } }
I20260923 18:04:16.421783 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.417956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41292 (local address 127.15.139.129:44163)
0923 18:04:16.418110 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:16.418114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:16.418128 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:04:16.418201 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:16.418203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:16.418257 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:16.418355 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:16.418568 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.419171 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.420169 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.420407 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.420502 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.420827 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:16.421296 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:16.421422 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:16.421538 (+ 116us) server_negotiation.cc:299] Negotiation successful
0923 18:04:16.421606 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 18:04:16.429672 1864 raft_consensus.cc:1261] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e [term 1 FOLLOWER]: Refusing update from remote peer 0be93de7aa664621b6cf41d3126e47f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:16.429725 1998 raft_consensus.cc:1261] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef [term 1 FOLLOWER]: Refusing update from remote peer 0be93de7aa664621b6cf41d3126e47f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:16.430449 2072 consensus_queue.cc:1035] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5966a18ccdbc449eacc02573cdafd38e" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 45139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:16.430754 2068 consensus_queue.cc:1035] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac5ccc8b5c81446a9797e2445c6787ef" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:16.440469 2076 mvcc.cc:205] Tried to move back new op lower bound from 7332604544728211456 to 7332604544596934656. Current Snapshot: MvccSnapshot[applied={T|T < 7332604544728211456}]
I20260923 18:04:16.449190 1584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:04:16.449417 1584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:04:16.452450 1584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7df906d1aed343338835849850f74170: Sending DeleteTablet for 3 replicas of tablet c93dd1663ef548db945a05b16b321ec5
I20260923 18:04:16.453418 1710 tablet_service.cc:1514] Processing DeleteTablet for tablet c93dd1663ef548db945a05b16b321ec5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:16 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:41276
I20260923 18:04:16.453439 1978 tablet_service.cc:1514] Processing DeleteTablet for tablet c93dd1663ef548db945a05b16b321ec5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:16 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:39690
I20260923 18:04:16.453437 1844 tablet_service.cc:1514] Processing DeleteTablet for tablet c93dd1663ef548db945a05b16b321ec5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:16 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:55040
I20260923 18:04:16.453989 2087 tablet_replica.cc:330] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef: stopping tablet replica
I20260923 18:04:16.454130 2085 tablet_replica.cc:330] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5: stopping tablet replica
I20260923 18:04:16.454447 2087 raft_consensus.cc:2229] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:04:16.454486 2085 raft_consensus.cc:2229] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:04:16.454864 2085 raft_consensus.cc:2258] T c93dd1663ef548db945a05b16b321ec5 P 0be93de7aa664621b6cf41d3126e47f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:16.454902 2087 raft_consensus.cc:2258] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:16.455024 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 1647
I20260923 18:04:16.457484 2086 tablet_replica.cc:330] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e: stopping tablet replica
I20260923 18:04:16.457846 2086 raft_consensus.cc:2229] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:04:16.458180 2086 raft_consensus.cc:2258] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:16.458663 2087 ts_tablet_manager.cc:1905] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:04:16.459600 2086 ts_tablet_manager.cc:1905] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:04:16.461758 2086 ts_tablet_manager.cc:1918] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:04:16.461881 2086 log.cc:1196] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/c93dd1663ef548db945a05b16b321ec5
I20260923 18:04:16.462221 2086 ts_tablet_manager.cc:1939] T c93dd1663ef548db945a05b16b321ec5 P 5966a18ccdbc449eacc02573cdafd38e: Deleting consensus metadata
I20260923 18:04:16.462618 2087 ts_tablet_manager.cc:1918] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:04:16.462767 2087 log.cc:1196] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/c93dd1663ef548db945a05b16b321ec5
I20260923 18:04:16.463160 2087 ts_tablet_manager.cc:1939] T c93dd1663ef548db945a05b16b321ec5 P ac5ccc8b5c81446a9797e2445c6787ef: Deleting consensus metadata
W20260923 18:04:16.463666 1570 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:44163 (error 108)
I20260923 18:04:16.463845 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 1781
I20260923 18:04:16.464821 1569 catalog_manager.cc:4841] TS 5966a18ccdbc449eacc02573cdafd38e (127.15.139.130:45139): tablet c93dd1663ef548db945a05b16b321ec5 (table test-table [id=28374436c3304c34849db32b0a321d99]) successfully deleted
I20260923 18:04:16.465237 1572 catalog_manager.cc:4841] TS ac5ccc8b5c81446a9797e2445c6787ef (127.15.139.131:43107): tablet c93dd1663ef548db945a05b16b321ec5 (table test-table [id=28374436c3304c34849db32b0a321d99]) successfully deleted
I20260923 18:04:16.465731 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.465340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:44163 (local address 127.0.0.1:41306)
0923 18:04:16.465541 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:16.465626 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:44163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260923 18:04:16.465914 1570 catalog_manager.cc:4584] TS 0be93de7aa664621b6cf41d3126e47f5 (127.15.139.129:44163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c93dd1663ef548db945a05b16b321ec5: Network error: Client connection negotiation failed: client connection to 127.15.139.129:44163: connect: Connection refused (error 111)
I20260923 18:04:16.470844 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 1915
I20260923 18:04:16.477210 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 1553
2026-09-23T18:04:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2303 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-23T18:04:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:16Z Disabled control of system clock
I20260923 18:04:16.509126 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36177
--webserver_interface=127.15.139.190
--webserver_port=0
--builtin_ntp_servers=127.15.139.148:33763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:36177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 18:04:16.620090 2092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:16.620388 2092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:16.620435 2092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:16.624490 2092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:16.624557 2092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:16.624574 2092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:16.624593 2092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:16.624608 2092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:16.629479 2092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:36177
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:36177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:16.630564 2092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:16.632009 2092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:16.640750 2100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:16.640919 2097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:16.640758 2098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:16.641093 2092 server_base.cc:1034] running on GCE node
I20260923 18:04:16.641892 2092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:16.643149 2092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:16.644383 2092 hybrid_clock.cc:648] HybridClock initialized: now 1790186656644306 us; error 91 us; skew 500 ppm
I20260923 18:04:16.646384 2092 webserver.cc:460] Webserver started at http://127.15.139.190:39587/ using document root <none> and password file <none>
I20260923 18:04:16.646993 2092 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:16.647089 2092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:16.647339 2092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:16.649477 2092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "8dc91ad925a84b39a309f71501511f26"
format_stamp: "Formatted at 2026-09-23 18:04:16 on dist-test-slave-vr57"
server_key: "a041dce1579a1ad5d35b01570b1c65b3"
server_key_iv: "2670aa552cf10ad7db420e61503dcd1d"
server_key_version: "encryptionkey@0"
I20260923 18:04:16.650058 2092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "8dc91ad925a84b39a309f71501511f26"
format_stamp: "Formatted at 2026-09-23 18:04:16 on dist-test-slave-vr57"
server_key: "a041dce1579a1ad5d35b01570b1c65b3"
server_key_iv: "2670aa552cf10ad7db420e61503dcd1d"
server_key_version: "encryptionkey@0"
I20260923 18:04:16.654004 2092 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:04:16.656649 2106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:16.658062 2092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:16.658218 2092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "8dc91ad925a84b39a309f71501511f26"
format_stamp: "Formatted at 2026-09-23 18:04:16 on dist-test-slave-vr57"
server_key: "a041dce1579a1ad5d35b01570b1c65b3"
server_key_iv: "2670aa552cf10ad7db420e61503dcd1d"
server_key_version: "encryptionkey@0"
I20260923 18:04:16.658344 2092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:16.668769 2092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:16.669435 2092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:16.669632 2092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:16.677031 2092 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:36177
I20260923 18:04:16.677042 2158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:36177 every 8 connection(s)
I20260923 18:04:16.677965 2092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:16.681133 2159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:16.685204 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 2092
I20260923 18:04:16.685353 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:16.685683 15918 external_mini_cluster.cc:1428] Setting key 8a6bf6cb7db030fff9712b7d21364f99
I20260923 18:04:16.687646 2159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26: Bootstrap starting.
I20260923 18:04:16.690069 2159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:16.690830 2159 log.cc:824] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:16.693102 2159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26: No bootstrap required, opened a new log
I20260923 18:04:16.695253 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.687272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46410 (local address 127.15.139.190:36177)
0923 18:04:16.687989 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:16.688006 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:16.688042 (+ 36us) server_negotiation.cc:407] Connection header received
0923 18:04:16.688704 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:16.688722 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:16.689115 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:16.689396 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:16.689751 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.690809 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.691987 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.692447 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.692586 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.693499 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:16.693503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:16.693517 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:16.693923 (+ 406us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:16.693976 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:16.693995 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:16.694074 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:16.694247 (+ 173us) server_negotiation.cc:299] Negotiation successful
0923 18:04:16.694531 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":174,"threads_started":1}
I20260923 18:04:16.697222 2159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dc91ad925a84b39a309f71501511f26" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36177 } }
I20260923 18:04:16.697523 2159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:16.697630 2159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc91ad925a84b39a309f71501511f26, State: Initialized, Role: FOLLOWER
I20260923 18:04:16.698112 2159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dc91ad925a84b39a309f71501511f26" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36177 } }
I20260923 18:04:16.698272 2159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:16.698367 2159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:16.698473 2159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:16.699457 2159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dc91ad925a84b39a309f71501511f26" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36177 } }
I20260923 18:04:16.699853 2159 leader_election.cc:302] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dc91ad925a84b39a309f71501511f26; no voters:
I20260923 18:04:16.700188 2159 leader_election.cc:288] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:16.700297 2164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:16.700553 2164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [term 1 LEADER]: Becoming Leader. State: Replica: 8dc91ad925a84b39a309f71501511f26, State: Running, Role: LEADER
I20260923 18:04:16.700958 2164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dc91ad925a84b39a309f71501511f26" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36177 } }
I20260923 18:04:16.701584 2159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:16.702731 2166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dc91ad925a84b39a309f71501511f26. Latest consensus state: current_term: 1 leader_uuid: "8dc91ad925a84b39a309f71501511f26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dc91ad925a84b39a309f71501511f26" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36177 } } }
I20260923 18:04:16.702862 2166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:16.703279 2173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:16.703280 2165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8dc91ad925a84b39a309f71501511f26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dc91ad925a84b39a309f71501511f26" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 36177 } } }
I20260923 18:04:16.704012 2165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:16.706528 2173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:16.712148 2173 catalog_manager.cc:1347] Generated new cluster ID: 2d2ee3c249794ebeaf4f27c0b2a0c44f
I20260923 18:04:16.712236 2173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:16.734090 2173 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:16.735073 2173 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:16.743332 2173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26: Generated new TSK 0
I20260923 18:04:16.744306 2173 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:16.748111 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36177
--builtin_ntp_servers=127.15.139.148:33763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 18:04:16.863826 2183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:16.864112 2183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:16.864153 2183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:16.868247 2183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:16.868312 2183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:16.868405 2183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:04:16.873268 2183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:16.874437 2183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:16.875963 2183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:16.885890 2188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:16.885915 2189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:16.886204 2183 server_base.cc:1034] running on GCE node
W20260923 18:04:16.885917 2191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:16.886804 2183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:16.887595 2183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:16.888835 2183 hybrid_clock.cc:648] HybridClock initialized: now 1790186656888810 us; error 41 us; skew 500 ppm
I20260923 18:04:16.891093 2183 webserver.cc:460] Webserver started at http://127.15.139.129:44047/ using document root <none> and password file <none>
I20260923 18:04:16.891824 2183 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:16.891922 2183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:16.892186 2183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:16.894378 2183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "d2cb706d77be4fafb55f9719df2d05f1"
format_stamp: "Formatted at 2026-09-23 18:04:16 on dist-test-slave-vr57"
server_key: "c2d8a516ab75dd7de45a908597e080b9"
server_key_iv: "97f7c68bae7b9e50fd9afe205f79a987"
server_key_version: "encryptionkey@0"
I20260923 18:04:16.894995 2183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "d2cb706d77be4fafb55f9719df2d05f1"
format_stamp: "Formatted at 2026-09-23 18:04:16 on dist-test-slave-vr57"
server_key: "c2d8a516ab75dd7de45a908597e080b9"
server_key_iv: "97f7c68bae7b9e50fd9afe205f79a987"
server_key_version: "encryptionkey@0"
I20260923 18:04:16.898867 2183 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260923 18:04:16.901526 2197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:16.903024 2183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260923 18:04:16.903156 2183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "d2cb706d77be4fafb55f9719df2d05f1"
format_stamp: "Formatted at 2026-09-23 18:04:16 on dist-test-slave-vr57"
server_key: "c2d8a516ab75dd7de45a908597e080b9"
server_key_iv: "97f7c68bae7b9e50fd9afe205f79a987"
server_key_version: "encryptionkey@0"
I20260923 18:04:16.903296 2183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:16.915767 2183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:16.916774 2183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:16.917050 2183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:16.917814 2183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:16.918891 2183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:16.918972 2183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:16.919049 2183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:16.919114 2183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:16.929469 2183 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:38971
I20260923 18:04:16.929541 2310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:38971 every 8 connection(s)
I20260923 18:04:16.930506 2183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:04:16.935267 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 2183
I20260923 18:04:16.935572 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:04:16.935923 15918 external_mini_cluster.cc:1428] Setting key e8f28f3c815ff757ce70baafbdcaaa93
I20260923 18:04:16.937999 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36177
--builtin_ntp_servers=127.15.139.148:33763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 18:04:16.940845 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.932626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:53355 (local address 127.15.139.190:36177)
0923 18:04:16.932806 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:16.932810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:16.933994 (+ 1184us) server_negotiation.cc:407] Connection header received
0923 18:04:16.935483 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:16.935487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:16.935585 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:16.935713 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:16.936735 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.937282 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.938588 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.938882 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.939015 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.940297 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:16.940300 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:16.940303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:16.940315 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:16.940337 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:16.940345 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:16.940443 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:16.940620 (+ 177us) server_negotiation.cc:299] Negotiation successful
0923 18:04:16.940685 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260923 18:04:16.942010 2313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:16.933084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36177 (local address 127.15.139.129:53355)
0923 18:04:16.933802 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:16.933847 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:16.935226 (+ 1379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:16.935907 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:16.935914 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:16.936001 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:16.936564 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.936579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.937435 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.937439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.938454 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:16.938465 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:16.939053 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:16.939057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:16.939235 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:16.940016 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:04:16.940018 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:16.940059 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:04:16.940123 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:16.940444 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:16.940456 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:16.940466 (+ 10us) client_negotiation.cc:769] Sending connection context
0923 18:04:16.940619 (+ 153us) client_negotiation.cc:240] Negotiation successful
0923 18:04:16.940918 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":123,"threads_started":1}
I20260923 18:04:16.943399 2311 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36177
I20260923 18:04:16.943749 2311 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:16.944312 2311 heartbeater.cc:511] Master 127.15.139.190:36177 requested a full tablet report, sending...
I20260923 18:04:16.945926 2123 ts_manager.cc:194] Registered new tserver with Master: d2cb706d77be4fafb55f9719df2d05f1 (127.15.139.129:38971)
I20260923 18:04:16.947577 2123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.139.129:53355
W20260923 18:04:17.056386 2314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:17.056656 2314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:17.056695 2314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:17.060792 2314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:17.060840 2314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:17.060925 2314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:04:17.065760 2314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:17.066862 2314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:17.068265 2314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:17.078271 2322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:17.078287 2320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:17.078389 2314 server_base.cc:1034] running on GCE node
W20260923 18:04:17.078271 2319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:17.079160 2314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:17.079813 2314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:17.080986 2314 hybrid_clock.cc:648] HybridClock initialized: now 1790186657080966 us; error 35 us; skew 500 ppm
I20260923 18:04:17.082978 2314 webserver.cc:460] Webserver started at http://127.15.139.130:37977/ using document root <none> and password file <none>
I20260923 18:04:17.083626 2314 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:17.083696 2314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:17.083940 2314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:17.085980 2314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "c8d6d49310f1462c98b3c31c0b58a4bf"
format_stamp: "Formatted at 2026-09-23 18:04:17 on dist-test-slave-vr57"
server_key: "22753769af70379185ee2d7ebab590bb"
server_key_iv: "d41490f5e29b967f22f46663391c6895"
server_key_version: "encryptionkey@0"
I20260923 18:04:17.086551 2314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "c8d6d49310f1462c98b3c31c0b58a4bf"
format_stamp: "Formatted at 2026-09-23 18:04:17 on dist-test-slave-vr57"
server_key: "22753769af70379185ee2d7ebab590bb"
server_key_iv: "d41490f5e29b967f22f46663391c6895"
server_key_version: "encryptionkey@0"
I20260923 18:04:17.090606 2314 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260923 18:04:17.093191 2328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:17.094484 2314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:04:17.094635 2314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "c8d6d49310f1462c98b3c31c0b58a4bf"
format_stamp: "Formatted at 2026-09-23 18:04:17 on dist-test-slave-vr57"
server_key: "22753769af70379185ee2d7ebab590bb"
server_key_iv: "d41490f5e29b967f22f46663391c6895"
server_key_version: "encryptionkey@0"
I20260923 18:04:17.094745 2314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:17.111248 2314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:17.112071 2314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:17.112286 2314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:17.112908 2314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:17.113914 2314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:17.113961 2314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:17.114032 2314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:17.114075 2314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:17.123895 2314 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:44981
I20260923 18:04:17.123987 2441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:44981 every 8 connection(s)
I20260923 18:04:17.124864 2314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:04:17.130236 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 2314
I20260923 18:04:17.130414 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:04:17.130774 15918 external_mini_cluster.cc:1428] Setting key 085f1d43855a1dbbafc40754909fba91
I20260923 18:04:17.132802 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36177
--builtin_ntp_servers=127.15.139.148:33763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 18:04:17.135473 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.127074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:47317 (local address 127.15.139.190:36177)
0923 18:04:17.127284 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:17.127289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:17.128519 (+ 1230us) server_negotiation.cc:407] Connection header received
0923 18:04:17.130009 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:17.130015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:17.130109 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:17.130223 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:17.131403 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.132049 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.133376 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.133618 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.133736 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.134913 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:17.134917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:17.134919 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:17.134932 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:17.134991 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:17.135001 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:17.135088 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:17.135260 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 18:04:17.135328 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260923 18:04:17.136476 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.127621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36177 (local address 127.15.139.130:47317)
0923 18:04:17.128338 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:17.128382 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:17.129751 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:17.130523 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:17.130542 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:17.130642 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:17.131239 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.131255 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.132202 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.132206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.133243 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.133253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.133784 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.133788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.133966 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.134611 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:04:17.134613 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:17.134656 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:04:17.134728 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:17.135090 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:17.135103 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:17.135112 (+ 9us) client_negotiation.cc:769] Sending connection context
0923 18:04:17.135258 (+ 146us) client_negotiation.cc:240] Negotiation successful
0923 18:04:17.135628 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":124,"threads_started":1}
I20260923 18:04:17.137863 2442 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36177
I20260923 18:04:17.138163 2442 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:17.138911 2442 heartbeater.cc:511] Master 127.15.139.190:36177 requested a full tablet report, sending...
I20260923 18:04:17.140331 2123 ts_manager.cc:194] Registered new tserver with Master: c8d6d49310f1462c98b3c31c0b58a4bf (127.15.139.130:44981)
I20260923 18:04:17.141252 2123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.139.130:47317
W20260923 18:04:17.248212 2445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:17.248543 2445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:17.248586 2445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:17.252579 2445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:17.252642 2445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:17.252696 2445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:04:17.257366 2445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:33763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:36177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:17.258411 2445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:17.259917 2445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:17.269680 2450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:17.269680 2451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:17.269692 2453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:17.270035 2445 server_base.cc:1034] running on GCE node
I20260923 18:04:17.270521 2445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:17.271248 2445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:17.272456 2445 hybrid_clock.cc:648] HybridClock initialized: now 1790186657272436 us; error 44 us; skew 500 ppm
I20260923 18:04:17.274461 2445 webserver.cc:460] Webserver started at http://127.15.139.131:39501/ using document root <none> and password file <none>
I20260923 18:04:17.275180 2445 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:17.275272 2445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:17.275516 2445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:17.278436 2445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "450926ce13e94a6cbba8791e63dbbbb3"
format_stamp: "Formatted at 2026-09-23 18:04:17 on dist-test-slave-vr57"
server_key: "f31918011eb9ab4697ea14e8544256a9"
server_key_iv: "75eb90f106081edb8cb7139b04ac0edf"
server_key_version: "encryptionkey@0"
I20260923 18:04:17.279117 2445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "450926ce13e94a6cbba8791e63dbbbb3"
format_stamp: "Formatted at 2026-09-23 18:04:17 on dist-test-slave-vr57"
server_key: "f31918011eb9ab4697ea14e8544256a9"
server_key_iv: "75eb90f106081edb8cb7139b04ac0edf"
server_key_version: "encryptionkey@0"
I20260923 18:04:17.283980 2445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260923 18:04:17.286577 2459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:17.288111 2445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:17.288275 2445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "450926ce13e94a6cbba8791e63dbbbb3"
format_stamp: "Formatted at 2026-09-23 18:04:17 on dist-test-slave-vr57"
server_key: "f31918011eb9ab4697ea14e8544256a9"
server_key_iv: "75eb90f106081edb8cb7139b04ac0edf"
server_key_version: "encryptionkey@0"
I20260923 18:04:17.288405 2445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:17.310050 2445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:17.310794 2445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:17.310968 2445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:17.311602 2445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:17.312675 2445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:17.312728 2445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:17.312773 2445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:17.312799 2445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:17.322062 2445 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:43559
I20260923 18:04:17.322126 2572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:43559 every 8 connection(s)
I20260923 18:04:17.323118 2445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:04:17.329845 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 2445
I20260923 18:04:17.329945 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:04:17.330220 15918 external_mini_cluster.cc:1428] Setting key d933322b3493816cbdc03ec27e687c83
I20260923 18:04:17.333258 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.325040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:35751 (local address 127.15.139.190:36177)
0923 18:04:17.325216 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:17.325220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:17.326315 (+ 1095us) server_negotiation.cc:407] Connection header received
0923 18:04:17.327887 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:17.327890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:17.327937 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:17.328014 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:17.328937 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.329590 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.330994 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.331336 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.331450 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.332802 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:17.332805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:17.332808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:17.332827 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:17.332846 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:17.332854 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:17.332913 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:17.333102 (+ 189us) server_negotiation.cc:299] Negotiation successful
0923 18:04:17.333151 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260923 18:04:17.334221 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.325451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:36177 (local address 127.15.139.131:35751)
0923 18:04:17.326146 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:17.326190 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:17.327684 (+ 1494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:17.328189 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:17.328196 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:17.328285 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:17.328806 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.328819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.329724 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.329727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.330854 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.330870 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.331556 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.331563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.331803 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.332472 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:04:17.332475 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:17.332540 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:04:17.332634 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:17.332932 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:17.332939 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:17.332944 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 18:04:17.333083 (+ 139us) client_negotiation.cc:240] Negotiation successful
0923 18:04:17.333344 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":113,"threads_started":1}
I20260923 18:04:17.335490 2573 heartbeater.cc:347] Connected to a master server at 127.15.139.190:36177
I20260923 18:04:17.335805 2573 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:17.336388 2573 heartbeater.cc:511] Master 127.15.139.190:36177 requested a full tablet report, sending...
I20260923 18:04:17.337538 2123 ts_manager.cc:194] Registered new tserver with Master: 450926ce13e94a6cbba8791e63dbbbb3 (127.15.139.131:43559)
I20260923 18:04:17.338130 2123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.139.131:35751
I20260923 18:04:17.344683 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:04:17.351220 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.346839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46416 (local address 127.15.139.190:36177)
0923 18:04:17.347016 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:17.347021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:17.347196 (+ 175us) server_negotiation.cc:407] Connection header received
0923 18:04:17.347363 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:17.347370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:17.347423 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:17.347508 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:17.347832 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.348352 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.349300 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.349526 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.349638 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.350590 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:17.350594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:17.350596 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:17.350608 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:17.350630 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:17.350639 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:17.350789 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:17.350967 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 18:04:17.351011 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260923 18:04:17.354328 2123 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:04:17.357710 2123 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:04:17.373328 2586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.366518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49894 (local address 127.15.139.131:43559)
0923 18:04:17.367165 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:17.367180 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:17.367850 (+ 670us) server_negotiation.cc:407] Connection header received
0923 18:04:17.368218 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:17.368226 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:17.368454 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:17.368623 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:17.368911 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.370121 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.371830 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.372378 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.372454 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.372584 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:17.373090 (+ 506us) server_negotiation.cc:299] Negotiation successful
0923 18:04:17.373188 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":170,"threads_started":1}
I20260923 18:04:17.373306 2588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.367259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:44981 (local address 127.0.0.1:33914)
0923 18:04:17.368146 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:17.368161 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:17.368259 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:17.368837 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:17.368840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:17.368870 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:17.369025 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.369031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.370126 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.370133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.371488 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.371548 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.372694 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.372698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.372856 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.372906 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 18:04:17.373017 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 18:04:17.373142 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":159,"spinlock_wait_cycles":7552,"thread_start_us":115,"threads_started":1}
I20260923 18:04:17.373572 2587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.366896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42622 (local address 127.15.139.129:38971)
0923 18:04:17.367416 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:17.367430 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:17.367853 (+ 423us) server_negotiation.cc:407] Connection header received
0923 18:04:17.368262 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:17.368269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:17.368494 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:17.368681 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:17.369161 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.370253 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.371869 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.372714 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.372833 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.372985 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:17.373267 (+ 282us) server_negotiation.cc:299] Negotiation successful
0923 18:04:17.373384 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":155,"threads_started":1}
I20260923 18:04:17.373741 2584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.366370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:38971 (local address 127.0.0.1:42622)
0923 18:04:17.367681 (+ 1311us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:17.367743 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:17.367987 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:17.368837 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:17.368840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:17.368864 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:17.369024 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.369031 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.370365 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.370371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.371739 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.371750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.373008 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.373014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.373107 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.373127 (+ 20us) client_negotiation.cc:769] Sending connection context
0923 18:04:17.373229 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 18:04:17.373311 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":31,"spinlock_wait_cycles":16640,"thread_start_us":95,"threads_started":1}
I20260923 18:04:17.374004 2589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.367478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33914 (local address 127.15.139.130:44981)
0923 18:04:17.367941 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:17.367954 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:17.368281 (+ 327us) server_negotiation.cc:407] Connection header received
0923 18:04:17.368362 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:17.368367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:17.368543 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:17.368681 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:17.369198 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.369991 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.371679 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.372551 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.372705 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.372902 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:17.373407 (+ 505us) server_negotiation.cc:299] Negotiation successful
0923 18:04:17.373511 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":136,"threads_started":1}
I20260923 18:04:17.374980 2585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.366342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:43559 (local address 127.0.0.1:49894)
0923 18:04:17.367681 (+ 1339us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:17.367743 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:17.367987 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:17.368619 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:17.368625 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:17.368647 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:17.368788 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.368796 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.370251 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.370255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.371683 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.371694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.372694 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.372698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.372853 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.372914 (+ 61us) client_negotiation.cc:769] Sending connection context
0923 18:04:17.374773 (+ 1859us) client_negotiation.cc:240] Negotiation successful
0923 18:04:17.374852 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1125,"mutex_wait_us":109,"spinlock_wait_cycles":9088,"thread_start_us":88,"threads_started":1}
I20260923 18:04:17.376828 2376 tablet_service.cc:1467] Processing CreateTablet for tablet b453788f8c93440e841b04ff525911d1 (DEFAULT_TABLE table=test-table [id=ad7de293c4084f5aa982590cc3b8505d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:17.376893 2245 tablet_service.cc:1467] Processing CreateTablet for tablet b453788f8c93440e841b04ff525911d1 (DEFAULT_TABLE table=test-table [id=ad7de293c4084f5aa982590cc3b8505d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:17.377396 2507 tablet_service.cc:1467] Processing CreateTablet for tablet b453788f8c93440e841b04ff525911d1 (DEFAULT_TABLE table=test-table [id=ad7de293c4084f5aa982590cc3b8505d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:17.377785 2376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b453788f8c93440e841b04ff525911d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:17.377898 2245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b453788f8c93440e841b04ff525911d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:17.378242 2507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b453788f8c93440e841b04ff525911d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:17.384447 2590 tablet_bootstrap.cc:492] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf: Bootstrap starting.
I20260923 18:04:17.385967 2592 tablet_bootstrap.cc:492] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1: Bootstrap starting.
I20260923 18:04:17.386011 2591 tablet_bootstrap.cc:492] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3: Bootstrap starting.
I20260923 18:04:17.387086 2590 tablet_bootstrap.cc:654] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:17.388099 2591 tablet_bootstrap.cc:654] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:17.388098 2592 tablet_bootstrap.cc:654] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:17.388092 2590 log.cc:824] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:17.388927 2591 log.cc:824] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:17.388928 2592 log.cc:824] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:17.390659 2590 tablet_bootstrap.cc:492] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf: No bootstrap required, opened a new log
I20260923 18:04:17.390869 2592 tablet_bootstrap.cc:492] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1: No bootstrap required, opened a new log
I20260923 18:04:17.390867 2591 tablet_bootstrap.cc:492] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3: No bootstrap required, opened a new log
I20260923 18:04:17.390893 2590 ts_tablet_manager.cc:1397] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260923 18:04:17.391052 2592 ts_tablet_manager.cc:1397] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260923 18:04:17.391052 2591 ts_tablet_manager.cc:1397] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260923 18:04:17.394618 2590 raft_consensus.cc:348] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "450926ce13e94a6cbba8791e63dbbbb3" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43559 } } peers { permanent_uuid: "d2cb706d77be4fafb55f9719df2d05f1" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38971 } } peers { permanent_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44981 } }
I20260923 18:04:17.394872 2590 raft_consensus.cc:374] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:17.394932 2590 raft_consensus.cc:729] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8d6d49310f1462c98b3c31c0b58a4bf, State: Initialized, Role: FOLLOWER
I20260923 18:04:17.395287 2592 raft_consensus.cc:348] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "450926ce13e94a6cbba8791e63dbbbb3" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43559 } } peers { permanent_uuid: "d2cb706d77be4fafb55f9719df2d05f1" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38971 } } peers { permanent_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44981 } }
I20260923 18:04:17.395287 2591 raft_consensus.cc:348] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "450926ce13e94a6cbba8791e63dbbbb3" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43559 } } peers { permanent_uuid: "d2cb706d77be4fafb55f9719df2d05f1" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38971 } } peers { permanent_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44981 } }
I20260923 18:04:17.395548 2591 raft_consensus.cc:374] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:17.395548 2592 raft_consensus.cc:374] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:17.395473 2590 consensus_queue.cc:260] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "450926ce13e94a6cbba8791e63dbbbb3" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43559 } } peers { permanent_uuid: "d2cb706d77be4fafb55f9719df2d05f1" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38971 } } peers { permanent_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44981 } }
I20260923 18:04:17.395613 2592 raft_consensus.cc:729] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2cb706d77be4fafb55f9719df2d05f1, State: Initialized, Role: FOLLOWER
I20260923 18:04:17.395622 2591 raft_consensus.cc:729] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 450926ce13e94a6cbba8791e63dbbbb3, State: Initialized, Role: FOLLOWER
I20260923 18:04:17.396082 2591 consensus_queue.cc:260] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "450926ce13e94a6cbba8791e63dbbbb3" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43559 } } peers { permanent_uuid: "d2cb706d77be4fafb55f9719df2d05f1" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38971 } } peers { permanent_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44981 } }
I20260923 18:04:17.396072 2592 consensus_queue.cc:260] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "450926ce13e94a6cbba8791e63dbbbb3" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43559 } } peers { permanent_uuid: "d2cb706d77be4fafb55f9719df2d05f1" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38971 } } peers { permanent_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44981 } }
I20260923 18:04:17.396682 2590 ts_tablet_manager.cc:1428] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260923 18:04:17.396999 2592 ts_tablet_manager.cc:1428] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260923 18:04:17.396999 2591 ts_tablet_manager.cc:1428] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260923 18:04:17.397079 2442 heartbeater.cc:503] Master 127.15.139.190:36177 was elected leader, sending a full tablet report...
I20260923 18:04:17.397326 2311 heartbeater.cc:503] Master 127.15.139.190:36177 was elected leader, sending a full tablet report...
I20260923 18:04:17.397326 2573 heartbeater.cc:503] Master 127.15.139.190:36177 was elected leader, sending a full tablet report...
W20260923 18:04:17.431777 2312 tablet.cc:2369] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:04:17.464010 2596 raft_consensus.cc:482] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:04:17.464275 2596 raft_consensus.cc:504] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "450926ce13e94a6cbba8791e63dbbbb3" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43559 } } peers { permanent_uuid: "d2cb706d77be4fafb55f9719df2d05f1" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38971 } } peers { permanent_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44981 } }
I20260923 18:04:17.465709 2596 leader_election.cc:288] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 450926ce13e94a6cbba8791e63dbbbb3 (127.15.139.131:43559), d2cb706d77be4fafb55f9719df2d05f1 (127.15.139.129:38971)
I20260923 18:04:17.471231 2587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.466099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:56695 (local address 127.15.139.129:38971)
0923 18:04:17.466242 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:17.466247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:17.466448 (+ 201us) server_negotiation.cc:407] Connection header received
0923 18:04:17.466540 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:17.466543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:17.466592 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:17.466676 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:17.467161 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.467995 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.470000 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.470593 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.470735 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.470790 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:17.471030 (+ 240us) server_negotiation.cc:299] Negotiation successful
0923 18:04:17.471100 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 18:04:17.471231 2586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.466305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:56353 (local address 127.15.139.131:43559)
0923 18:04:17.466470 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:17.466476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:17.466492 (+ 16us) server_negotiation.cc:407] Connection header received
0923 18:04:17.466539 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:17.466542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:17.466592 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:17.466676 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:17.467159 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.468035 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.470000 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.470674 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.470777 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.470833 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:17.471030 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 18:04:17.471100 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 18:04:17.471263 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.465933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:43559 (local address 127.15.139.130:56353)
0923 18:04:17.466225 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:17.466242 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:17.466335 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:17.466703 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:17.466706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:17.466747 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:17.467014 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.467021 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.468139 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.468142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.469856 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.469868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.470788 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.470791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.470905 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.470922 (+ 17us) client_negotiation.cc:769] Sending connection context
0923 18:04:17.470988 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 18:04:17.471100 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":365,"spinlock_wait_cycles":6912}
I20260923 18:04:17.471272 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.465948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:38971 (local address 127.15.139.130:56695)
0923 18:04:17.466345 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:17.466356 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:17.466447 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:17.466703 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:17.466706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:17.466746 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:17.467015 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.467021 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.468124 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.468129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.469856 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:17.469868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.470744 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.470750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:17.470853 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.470904 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 18:04:17.470988 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 18:04:17.471092 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":274,"spinlock_wait_cycles":27904,"thread_start_us":117,"threads_started":1}
I20260923 18:04:17.471942 2527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b453788f8c93440e841b04ff525911d1" candidate_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "450926ce13e94a6cbba8791e63dbbbb3" is_pre_election: true
I20260923 18:04:17.471990 2265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b453788f8c93440e841b04ff525911d1" candidate_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2cb706d77be4fafb55f9719df2d05f1" is_pre_election: true
I20260923 18:04:17.472258 2527 raft_consensus.cc:2454] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8d6d49310f1462c98b3c31c0b58a4bf in term 0.
I20260923 18:04:17.472287 2265 raft_consensus.cc:2454] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8d6d49310f1462c98b3c31c0b58a4bf in term 0.
I20260923 18:04:17.472779 2329 leader_election.cc:302] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 450926ce13e94a6cbba8791e63dbbbb3, c8d6d49310f1462c98b3c31c0b58a4bf; no voters:
I20260923 18:04:17.473060 2596 raft_consensus.cc:2781] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:04:17.473170 2596 raft_consensus.cc:482] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:04:17.473232 2596 raft_consensus.cc:3037] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:17.474256 2596 raft_consensus.cc:504] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "450926ce13e94a6cbba8791e63dbbbb3" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43559 } } peers { permanent_uuid: "d2cb706d77be4fafb55f9719df2d05f1" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38971 } } peers { permanent_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44981 } }
I20260923 18:04:17.474635 2596 leader_election.cc:288] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [CANDIDATE]: Term 1 election: Requested vote from peers 450926ce13e94a6cbba8791e63dbbbb3 (127.15.139.131:43559), d2cb706d77be4fafb55f9719df2d05f1 (127.15.139.129:38971)
I20260923 18:04:17.474985 2527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b453788f8c93440e841b04ff525911d1" candidate_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "450926ce13e94a6cbba8791e63dbbbb3"
I20260923 18:04:17.475003 2265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b453788f8c93440e841b04ff525911d1" candidate_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2cb706d77be4fafb55f9719df2d05f1"
I20260923 18:04:17.475119 2527 raft_consensus.cc:3037] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:17.475152 2265 raft_consensus.cc:3037] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:17.476222 2265 raft_consensus.cc:2454] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8d6d49310f1462c98b3c31c0b58a4bf in term 1.
I20260923 18:04:17.476225 2527 raft_consensus.cc:2454] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8d6d49310f1462c98b3c31c0b58a4bf in term 1.
I20260923 18:04:17.476524 2329 leader_election.cc:302] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 450926ce13e94a6cbba8791e63dbbbb3, c8d6d49310f1462c98b3c31c0b58a4bf; no voters:
I20260923 18:04:17.476699 2596 raft_consensus.cc:2781] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:17.476904 2596 raft_consensus.cc:686] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 1 LEADER]: Becoming Leader. State: Replica: c8d6d49310f1462c98b3c31c0b58a4bf, State: Running, Role: LEADER
I20260923 18:04:17.477250 2596 consensus_queue.cc:237] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "450926ce13e94a6cbba8791e63dbbbb3" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43559 } } peers { permanent_uuid: "d2cb706d77be4fafb55f9719df2d05f1" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38971 } } peers { permanent_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44981 } }
I20260923 18:04:17.480268 2123 catalog_manager.cc:5495] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf reported cstate change: term changed from 0 to 1, leader changed from <none> to c8d6d49310f1462c98b3c31c0b58a4bf (127.15.139.130). New cstate: current_term: 1 leader_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "450926ce13e94a6cbba8791e63dbbbb3" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2cb706d77be4fafb55f9719df2d05f1" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8d6d49310f1462c98b3c31c0b58a4bf" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 44981 } health_report { overall_health: HEALTHY } } }
I20260923 18:04:17.496399 2589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.493126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.15.139.130:44981)
0923 18:04:17.493255 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:17.493259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:17.493288 (+ 29us) server_negotiation.cc:407] Connection header received
0923 18:04:17.493359 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:17.493361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:17.493415 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:17.493501 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:17.493731 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.494240 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.495039 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:17.495276 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:17.495353 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:17.495672 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:17.496072 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:17.496181 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:17.496257 (+ 76us) server_negotiation.cc:299] Negotiation successful
0923 18:04:17.496289 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 18:04:17.502537 2527 raft_consensus.cc:1261] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3 [term 1 FOLLOWER]: Refusing update from remote peer c8d6d49310f1462c98b3c31c0b58a4bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:17.502714 2265 raft_consensus.cc:1261] T b453788f8c93440e841b04ff525911d1 P d2cb706d77be4fafb55f9719df2d05f1 [term 1 FOLLOWER]: Refusing update from remote peer c8d6d49310f1462c98b3c31c0b58a4bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:17.503273 2596 consensus_queue.cc:1035] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "450926ce13e94a6cbba8791e63dbbbb3" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 43559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:17.503554 2600 consensus_queue.cc:1035] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2cb706d77be4fafb55f9719df2d05f1" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:17.514336 2604 mvcc.cc:205] Tried to move back new op lower bound from 7332604549122809856 to 7332604549029351424. Current Snapshot: MvccSnapshot[applied={T|T < 7332604549122809856}]
I20260923 18:04:17.524552 2122 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:04:17.524767 2122 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:04:17.527980 2122 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8dc91ad925a84b39a309f71501511f26: Sending DeleteTablet for 3 replicas of tablet b453788f8c93440e841b04ff525911d1
I20260923 18:04:17.528898 2507 tablet_service.cc:1514] Processing DeleteTablet for tablet b453788f8c93440e841b04ff525911d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:17 UTC) from {username='slave'} at 127.0.0.1:49894
I20260923 18:04:17.529104 2376 tablet_service.cc:1514] Processing DeleteTablet for tablet b453788f8c93440e841b04ff525911d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:17 UTC) from {username='slave'} at 127.0.0.1:33914
I20260923 18:04:17.529408 2613 tablet_replica.cc:330] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3: stopping tablet replica
I20260923 18:04:17.529774 2613 raft_consensus.cc:2229] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:04:17.530108 2613 raft_consensus.cc:2258] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:17.530349 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 2183
I20260923 18:04:17.535862 2614 tablet_replica.cc:330] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf: stopping tablet replica
I20260923 18:04:17.536273 2614 raft_consensus.cc:2229] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:04:17.536809 2614 raft_consensus.cc:2258] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:17.537884 2613 ts_tablet_manager.cc:1905] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:04:17.538174 2614 ts_tablet_manager.cc:1905] T b453788f8c93440e841b04ff525911d1 P c8d6d49310f1462c98b3c31c0b58a4bf: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 18:04:17.539777 2108 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:38971 (error 108)
I20260923 18:04:17.540745 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 2314
I20260923 18:04:17.540899 2613 ts_tablet_manager.cc:1918] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:04:17.541437 2613 log.cc:1196] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/b453788f8c93440e841b04ff525911d1
I20260923 18:04:17.541759 2613 ts_tablet_manager.cc:1939] T b453788f8c93440e841b04ff525911d1 P 450926ce13e94a6cbba8791e63dbbbb3: Deleting consensus metadata
I20260923 18:04:17.542994 2107 catalog_manager.cc:4841] TS 450926ce13e94a6cbba8791e63dbbbb3 (127.15.139.131:43559): tablet b453788f8c93440e841b04ff525911d1 (table test-table [id=ad7de293c4084f5aa982590cc3b8505d]) successfully deleted
I20260923 18:04:17.543421 2585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.543127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:38971 (local address 127.0.0.1:42624)
0923 18:04:17.543290 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:17.543350 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260923 18:04:17.543599 2108 catalog_manager.cc:4584] TS d2cb706d77be4fafb55f9719df2d05f1 (127.15.139.129:38971): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b453788f8c93440e841b04ff525911d1: Network error: Client connection negotiation failed: client connection to 127.15.139.129:38971: connect: Connection refused (error 111)
I20260923 18:04:17.548341 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 2445
I20260923 18:04:17.549525 2585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:17.548508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:44981 (local address 127.0.0.1:33940)
0923 18:04:17.549358 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:17.549425 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.130:44981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":747}
I20260923 18:04:17.554167 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 2092
2026-09-23T18:04:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1075 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:17 dist-test-slave-vr57 krb5kdc[2618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:17 dist-test-slave-vr57 krb5kdc[2618](info): set up 2 sockets
Sep 23 18:04:17 dist-test-slave-vr57 krb5kdc[2618](info): commencing operation
krb5kdc: starting...
W20260923 18:04:18.596225 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:18 dist-test-slave-vr57 krb5kdc[2618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:04:18.722568 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:42615
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:45265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:42615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:18.836862 2631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:18.837150 2631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:18.837193 2631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:18.841331 2631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:18.841391 2631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:18.841411 2631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:18.841428 2631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:18.841481 2631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:18.846836 2631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:45265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:42615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:42615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:18.848147 2631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:18.849229 2631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:18.857870 2637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:18.857874 2639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:18.857890 2636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:18.858263 2631 server_base.cc:1034] running on GCE node
I20260923 18:04:18.858814 2631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:18.860047 2631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:18.861264 2631 hybrid_clock.cc:648] HybridClock initialized: now 1790186658861229 us; error 47 us; skew 500 ppm
Sep 23 18:04:18 dist-test-slave-vr57 krb5kdc[2618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186658, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:18.864542 2631 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:04:18.865864 2631 webserver.cc:460] Webserver started at http://127.15.139.190:41265/ using document root <none> and password file <none>
I20260923 18:04:18.866540 2631 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:18.866643 2631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:18.866885 2631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:18.869297 2631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "acd1c1b84e4746638049bae91574a058"
format_stamp: "Formatted at 2026-09-23 18:04:18 on dist-test-slave-vr57"
server_key: "888a18170fd37d94a8c741ef0c2423c9"
server_key_iv: "723dbfc7e2d6bb9c595da4a01f17a5ce"
server_key_version: "encryptionkey@0"
I20260923 18:04:18.869931 2631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "acd1c1b84e4746638049bae91574a058"
format_stamp: "Formatted at 2026-09-23 18:04:18 on dist-test-slave-vr57"
server_key: "888a18170fd37d94a8c741ef0c2423c9"
server_key_iv: "723dbfc7e2d6bb9c595da4a01f17a5ce"
server_key_version: "encryptionkey@0"
I20260923 18:04:18.874092 2631 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260923 18:04:18.876922 2646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:18.878402 2631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:04:18.878623 2631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "acd1c1b84e4746638049bae91574a058"
format_stamp: "Formatted at 2026-09-23 18:04:18 on dist-test-slave-vr57"
server_key: "888a18170fd37d94a8c741ef0c2423c9"
server_key_iv: "723dbfc7e2d6bb9c595da4a01f17a5ce"
server_key_version: "encryptionkey@0"
I20260923 18:04:18.878750 2631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:18.917264 2631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:18.920783 2631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:18.921007 2631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:18.928133 2631 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:42615
I20260923 18:04:18.928215 2698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:42615 every 8 connection(s)
I20260923 18:04:18.929147 2631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:18.929589 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 2631
I20260923 18:04:18.929741 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:18.930027 15918 external_mini_cluster.cc:1428] Setting key a2a0323d25f957be82ed6bc5260e09e3
I20260923 18:04:18.932668 2699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:18.939173 2699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058: Bootstrap starting.
Sep 23 18:04:18 dist-test-slave-vr57 krb5kdc[2618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:18.941977 2699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:18.942864 2699 log.cc:824] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:18.945197 2699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058: No bootstrap required, opened a new log
I20260923 18:04:18.945578 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:18.931436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34656 (local address 127.15.139.190:42615)
0923 18:04:18.931949 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:18.931968 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:18.932013 (+ 45us) server_negotiation.cc:407] Connection header received
0923 18:04:18.932899 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:18.932921 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:18.933348 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:18.933723 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:18.934419 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:18.935931 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:18.936791 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:18.937462 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:18.937639 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:18.940385 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:18.940392 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:18.940409 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:18.940444 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:18.942445 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:18.942890 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:18.942896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:18.942902 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:18.943038 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:18.943378 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:18.943381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:18.943382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:18.943895 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:18.944087 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:18.944340 (+ 253us) server_negotiation.cc:299] Negotiation successful
0923 18:04:18.944629 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":126,"threads_started":1}
I20260923 18:04:18.949136 2699 raft_consensus.cc:348] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd1c1b84e4746638049bae91574a058" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42615 } }
I20260923 18:04:18.949409 2699 raft_consensus.cc:374] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:18.949493 2699 raft_consensus.cc:729] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acd1c1b84e4746638049bae91574a058, State: Initialized, Role: FOLLOWER
I20260923 18:04:18.949970 2699 consensus_queue.cc:260] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd1c1b84e4746638049bae91574a058" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42615 } }
I20260923 18:04:18.950127 2699 raft_consensus.cc:388] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:18.950197 2699 raft_consensus.cc:482] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:18.950299 2699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:18.951233 2699 raft_consensus.cc:504] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd1c1b84e4746638049bae91574a058" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42615 } }
I20260923 18:04:18.951647 2699 leader_election.cc:302] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acd1c1b84e4746638049bae91574a058; no voters:
I20260923 18:04:18.951974 2699 leader_election.cc:288] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:18.952136 2704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:18.952411 2704 raft_consensus.cc:686] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [term 1 LEADER]: Becoming Leader. State: Replica: acd1c1b84e4746638049bae91574a058, State: Running, Role: LEADER
I20260923 18:04:18.952764 2704 consensus_queue.cc:237] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd1c1b84e4746638049bae91574a058" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42615 } }
I20260923 18:04:18.953275 2699 sys_catalog.cc:564] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:18.954618 2706 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acd1c1b84e4746638049bae91574a058. Latest consensus state: current_term: 1 leader_uuid: "acd1c1b84e4746638049bae91574a058" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd1c1b84e4746638049bae91574a058" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42615 } } }
I20260923 18:04:18.954643 2705 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acd1c1b84e4746638049bae91574a058" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd1c1b84e4746638049bae91574a058" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 42615 } } }
I20260923 18:04:18.954751 2706 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:18.954751 2705 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:18.955372 2714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:18.958318 2714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:18.963995 2714 catalog_manager.cc:1347] Generated new cluster ID: 6a976d654df5418fbc5ea044d4efb93e
I20260923 18:04:18.964083 2714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:18.976243 2714 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:18.977837 2714 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:18.991914 2714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058: Generated new TSK 0
I20260923 18:04:18.992939 2714 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.139.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
Entry for principal kudu/127.15.139.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab.
I20260923 18:04:19.022984 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42615
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:45265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:19.137353 2725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:19.137706 2725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:19.137753 2725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:19.142016 2725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:19.142107 2725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:19.142171 2725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:04:19.148185 2725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:45265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.129.keytab
--principal=kudu/127.15.139.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:19.149624 2725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:19.150830 2725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:19.162147 2730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:19.162151 2733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:19.162460 2725 server_base.cc:1034] running on GCE node
W20260923 18:04:19.162147 2731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:19.162953 2725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:19.163769 2725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:19.164973 2725 hybrid_clock.cc:648] HybridClock initialized: now 1790186659164927 us; error 52 us; skew 500 ppm
Sep 23 18:04:19 dist-test-slave-vr57 krb5kdc[2618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186659, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:19.172263 2725 init.cc:377] Logged in from keytab as kudu/127.15.139.129@KRBTEST.COM (short username kudu)
I20260923 18:04:19.173353 2725 webserver.cc:460] Webserver started at http://127.15.139.129:33555/ using document root <none> and password file <none>
I20260923 18:04:19.173957 2725 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:19.174054 2725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:19.174294 2725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:19.176425 2725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "441ed151e38b4ab1a4a185bdbd703284"
format_stamp: "Formatted at 2026-09-23 18:04:19 on dist-test-slave-vr57"
server_key: "cc1524a5af2352194b678ba81224bf27"
server_key_iv: "1e735608d50d9aef3e377d79a2813573"
server_key_version: "encryptionkey@0"
I20260923 18:04:19.176994 2725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "441ed151e38b4ab1a4a185bdbd703284"
format_stamp: "Formatted at 2026-09-23 18:04:19 on dist-test-slave-vr57"
server_key: "cc1524a5af2352194b678ba81224bf27"
server_key_iv: "1e735608d50d9aef3e377d79a2813573"
server_key_version: "encryptionkey@0"
I20260923 18:04:19.180698 2725 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260923 18:04:19.183900 2740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:19.185822 2725 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260923 18:04:19.185973 2725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "441ed151e38b4ab1a4a185bdbd703284"
format_stamp: "Formatted at 2026-09-23 18:04:19 on dist-test-slave-vr57"
server_key: "cc1524a5af2352194b678ba81224bf27"
server_key_iv: "1e735608d50d9aef3e377d79a2813573"
server_key_version: "encryptionkey@0"
I20260923 18:04:19.186110 2725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:19.245093 2725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:19.248302 2725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:19.248502 2725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:19.249138 2725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:19.249996 2725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:19.250067 2725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:19.250135 2725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:19.250182 2725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:19.259907 2725 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:38529
I20260923 18:04:19.259925 2853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:38529 every 8 connection(s)
I20260923 18:04:19.260876 2725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
Sep 23 18:04:19 dist-test-slave-vr57 krb5kdc[2618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186659, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.129@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:19.271031 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 2725
I20260923 18:04:19.271194 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:04:19.271548 15918 external_mini_cluster.cc:1428] Setting key e63f0e8f85097833614da182380e950d
I20260923 18:04:19.275568 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.262745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:47805 (local address 127.15.139.190:42615)
0923 18:04:19.262900 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:19.262905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:19.263843 (+ 938us) server_negotiation.cc:407] Connection header received
0923 18:04:19.265222 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:19.265226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:19.265273 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:19.265349 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:19.266451 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.267032 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.268281 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.268537 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.268626 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.271066 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:19.271069 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:19.271071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:19.271095 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:19.273062 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:19.273689 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:19.273692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:19.273693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:19.273772 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:19.274176 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:19.274178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:19.274179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:19.274348 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:19.274423 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:19.275161 (+ 738us) server_negotiation.cc:299] Negotiation successful
0923 18:04:19.275336 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 18:04:19.276369 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.263193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42615 (local address 127.15.139.129:47805)
0923 18:04:19.263691 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:19.263736 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:19.265012 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:19.265479 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:19.265485 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:19.265845 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:19.266316 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.266329 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.267162 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.267164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.268159 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.268169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.268679 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.268681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.268823 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.269340 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:19.269356 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:19.270884 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:19.273185 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:19.273191 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:19.273205 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:19.273578 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:19.273865 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:19.273871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:19.273876 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:19.274077 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:19.274452 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:19.274461 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:19.274850 (+ 389us) client_negotiation.cc:769] Sending connection context
0923 18:04:19.275093 (+ 243us) client_negotiation.cc:240] Negotiation successful
0923 18:04:19.275409 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":115,"threads_started":1}
I20260923 18:04:19.277726 2854 heartbeater.cc:347] Connected to a master server at 127.15.139.190:42615
I20260923 18:04:19.277956 2854 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:19.278440 2854 heartbeater.cc:511] Master 127.15.139.190:42615 requested a full tablet report, sending...
I20260923 18:04:19.279955 2663 ts_manager.cc:194] Registered new tserver with Master: 441ed151e38b4ab1a4a185bdbd703284 (127.15.139.129:38529)
I20260923 18:04:19.281081 2663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.129@KRBTEST.COM'} at 127.15.139.129:47805
WARNING: no policy specified for kudu/127.15.139.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
Entry for principal kudu/127.15.139.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab.
I20260923 18:04:19.299273 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42615
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:45265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:19.411625 2859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:19.411899 2859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:19.411947 2859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:19.415911 2859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:19.415967 2859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:19.416065 2859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:04:19.422739 2859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:45265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.130.keytab
--principal=kudu/127.15.139.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:19.424053 2859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:19.425129 2859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:19.435045 2867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:19.435045 2864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:19.435189 2865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:19.435564 2859 server_base.cc:1034] running on GCE node
I20260923 18:04:19.436005 2859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:19.436618 2859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:19.437785 2859 hybrid_clock.cc:648] HybridClock initialized: now 1790186659437769 us; error 30 us; skew 500 ppm
Sep 23 18:04:19 dist-test-slave-vr57 krb5kdc[2618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186659, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:19.440644 2859 init.cc:377] Logged in from keytab as kudu/127.15.139.130@KRBTEST.COM (short username kudu)
I20260923 18:04:19.441519 2859 webserver.cc:460] Webserver started at http://127.15.139.130:46429/ using document root <none> and password file <none>
I20260923 18:04:19.442131 2859 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:19.442227 2859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:19.442461 2859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:19.444634 2859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "916a62e93d2247f0824309c2ca8a9a8f"
format_stamp: "Formatted at 2026-09-23 18:04:19 on dist-test-slave-vr57"
server_key: "45e60e3545ba055f1785a8bf0c2a8cfa"
server_key_iv: "4994f9d2c4f2f483e60d5e44897fb4c5"
server_key_version: "encryptionkey@0"
I20260923 18:04:19.445326 2859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "916a62e93d2247f0824309c2ca8a9a8f"
format_stamp: "Formatted at 2026-09-23 18:04:19 on dist-test-slave-vr57"
server_key: "45e60e3545ba055f1785a8bf0c2a8cfa"
server_key_iv: "4994f9d2c4f2f483e60d5e44897fb4c5"
server_key_version: "encryptionkey@0"
I20260923 18:04:19.449095 2859 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260923 18:04:19.451462 2874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:19.452718 2859 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:04:19.452857 2859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "916a62e93d2247f0824309c2ca8a9a8f"
format_stamp: "Formatted at 2026-09-23 18:04:19 on dist-test-slave-vr57"
server_key: "45e60e3545ba055f1785a8bf0c2a8cfa"
server_key_iv: "4994f9d2c4f2f483e60d5e44897fb4c5"
server_key_version: "encryptionkey@0"
I20260923 18:04:19.452962 2859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:19.469714 2859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:19.472549 2859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:19.472759 2859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:19.473361 2859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:19.474191 2859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:19.474258 2859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:19.474328 2859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:19.474380 2859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:19.483384 2859 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:43875
I20260923 18:04:19.483428 2987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:43875 every 8 connection(s)
I20260923 18:04:19.484284 2859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:04:19.485823 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 2859
I20260923 18:04:19.485955 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:04:19.486270 15918 external_mini_cluster.cc:1428] Setting key 6fcc241f6f902f753daf82952600a6d0
Sep 23 18:04:19 dist-test-slave-vr57 krb5kdc[2618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186659, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.130@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.131@KRBTEST.COM" created.
I20260923 18:04:19.499789 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.486105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:47009 (local address 127.15.139.190:42615)
0923 18:04:19.486244 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:19.486249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:19.487390 (+ 1141us) server_negotiation.cc:407] Connection header received
0923 18:04:19.488842 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:19.488845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:19.488896 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:19.488973 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:19.490602 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.491248 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.492642 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.493002 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.493120 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.496101 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:19.496104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:19.496106 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:19.496129 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:19.497608 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:19.498190 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:19.498194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:19.498195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:19.498293 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:19.498668 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:19.498672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:19.498673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:19.498850 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:19.498955 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:19.499446 (+ 491us) server_negotiation.cc:299] Negotiation successful
0923 18:04:19.499615 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 18:04:19.500643 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.486622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42615 (local address 127.15.139.130:47009)
0923 18:04:19.487222 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:19.487280 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:19.488636 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:19.489191 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:19.489202 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:19.489762 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:19.490448 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.490463 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.491378 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.491381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.492517 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.492527 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.493219 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.493224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.493471 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.494128 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:19.494154 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:19.495921 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:19.497790 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:19.497796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:19.497812 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:19.498083 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:19.498382 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:19.498386 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:19.498388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:19.498554 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:19.498954 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:19.498960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:19.499210 (+ 250us) client_negotiation.cc:769] Sending connection context
0923 18:04:19.499392 (+ 182us) client_negotiation.cc:240] Negotiation successful
0923 18:04:19.499741 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":159,"threads_started":1}
I20260923 18:04:19.501874 2988 heartbeater.cc:347] Connected to a master server at 127.15.139.190:42615
I20260923 18:04:19.502140 2988 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:19.502825 2988 heartbeater.cc:511] Master 127.15.139.190:42615 requested a full tablet report, sending...
I20260923 18:04:19.504043 2663 ts_manager.cc:194] Registered new tserver with Master: 916a62e93d2247f0824309c2ca8a9a8f (127.15.139.130:43875)
I20260923 18:04:19.504921 2663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.130@KRBTEST.COM'} at 127.15.139.130:47009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
Entry for principal kudu/127.15.139.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab.
I20260923 18:04:19.515836 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42615
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:45265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:19.627053 2993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:19.627336 2993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:19.627379 2993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:19.631471 2993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:19.631547 2993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:19.631642 2993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:04:19.636761 2993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:45265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.131.keytab
--principal=kudu/127.15.139.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:42615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:19.637800 2993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:19.638842 2993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:19.648725 3001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:19.648815 2998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:19.648818 2999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:19.649106 2993 server_base.cc:1034] running on GCE node
I20260923 18:04:19.649521 2993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:19.650254 2993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:19.651459 2993 hybrid_clock.cc:648] HybridClock initialized: now 1790186659651439 us; error 41 us; skew 500 ppm
Sep 23 18:04:19 dist-test-slave-vr57 krb5kdc[2618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186659, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:19.654698 2993 init.cc:377] Logged in from keytab as kudu/127.15.139.131@KRBTEST.COM (short username kudu)
I20260923 18:04:19.655855 2993 webserver.cc:460] Webserver started at http://127.15.139.131:45467/ using document root <none> and password file <none>
I20260923 18:04:19.656587 2993 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:19.656687 2993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:19.656932 2993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:19.659258 2993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "c4ababc5f6b54151b5eff53c28deb664"
format_stamp: "Formatted at 2026-09-23 18:04:19 on dist-test-slave-vr57"
server_key: "abbcfc2ccabd22428227fe8214d282eb"
server_key_iv: "02bf49e233b697eb5cefb868ee205004"
server_key_version: "encryptionkey@0"
I20260923 18:04:19.659981 2993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "c4ababc5f6b54151b5eff53c28deb664"
format_stamp: "Formatted at 2026-09-23 18:04:19 on dist-test-slave-vr57"
server_key: "abbcfc2ccabd22428227fe8214d282eb"
server_key_iv: "02bf49e233b697eb5cefb868ee205004"
server_key_version: "encryptionkey@0"
I20260923 18:04:19.664057 2993 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:04:19.666603 3008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:19.668047 2993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:19.668210 2993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "c4ababc5f6b54151b5eff53c28deb664"
format_stamp: "Formatted at 2026-09-23 18:04:19 on dist-test-slave-vr57"
server_key: "abbcfc2ccabd22428227fe8214d282eb"
server_key_iv: "02bf49e233b697eb5cefb868ee205004"
server_key_version: "encryptionkey@0"
I20260923 18:04:19.668327 2993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:19.699846 2993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:19.703487 2993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:19.703779 2993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:19.704480 2993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:19.705408 2993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:19.705482 2993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:19.705582 2993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:19.705631 2993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:19.715606 2993 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:40505
I20260923 18:04:19.715616 3121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:40505 every 8 connection(s)
I20260923 18:04:19.716560 2993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:04:19.722884 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 2993
I20260923 18:04:19.723045 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:04:19.723345 15918 external_mini_cluster.cc:1428] Setting key 8196d606e0970868a80dd4a83ef8a8c1
Sep 23 18:04:19 dist-test-slave-vr57 krb5kdc[2618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186659, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.131@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:19.731153 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.718585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:46411 (local address 127.15.139.190:42615)
0923 18:04:19.718735 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:19.718739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:19.719685 (+ 946us) server_negotiation.cc:407] Connection header received
0923 18:04:19.721185 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:19.721188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:19.721248 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:19.721331 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:19.722526 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.723120 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.724621 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.724933 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.725055 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.727762 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:19.727765 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:19.727768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:19.727803 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:19.729030 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:19.729564 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:19.729567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:19.729569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:19.729657 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:19.730021 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:19.730024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:19.730026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:19.730226 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:19.730337 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:19.730851 (+ 514us) server_negotiation.cc:299] Negotiation successful
0923 18:04:19.730988 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260923 18:04:19.732141 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.718876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:42615 (local address 127.15.139.131:46411)
0923 18:04:19.719426 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:19.719476 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:19.720934 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:19.721506 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:19.721514 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:19.721870 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:19.722359 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.722373 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.723265 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.723267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.724490 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.724500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.725090 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.725094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.725279 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.725926 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:19.725943 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:19.727591 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:19.729157 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:19.729163 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:19.729179 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:19.729448 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:19.729753 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:19.729757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:19.729759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:19.729909 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:19.730335 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:19.730342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:19.730603 (+ 261us) client_negotiation.cc:769] Sending connection context
0923 18:04:19.730793 (+ 190us) client_negotiation.cc:240] Negotiation successful
0923 18:04:19.731153 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":157,"threads_started":1}
I20260923 18:04:19.733445 3122 heartbeater.cc:347] Connected to a master server at 127.15.139.190:42615
I20260923 18:04:19.733731 3122 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:19.734256 3122 heartbeater.cc:511] Master 127.15.139.190:42615 requested a full tablet report, sending...
I20260923 18:04:19.735348 2663 ts_manager.cc:194] Registered new tserver with Master: c4ababc5f6b54151b5eff53c28deb664 (127.15.139.131:40505)
I20260923 18:04:19.735976 2663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.139.131@KRBTEST.COM'} at 127.15.139.131:46411
I20260923 18:04:19.737694 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:04:19.747763 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.739788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34664 (local address 127.15.139.190:42615)
0923 18:04:19.739939 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:19.739944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:19.740105 (+ 161us) server_negotiation.cc:407] Connection header received
0923 18:04:19.740215 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:19.740218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:19.740259 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:19.740331 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:19.741049 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.741626 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.742528 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.742743 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.742831 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.744368 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:19.744371 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:19.744373 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:19.744469 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:19.745880 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:19.746318 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:19.746321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:19.746322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:19.746450 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:19.746751 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:19.746753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:19.746754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:19.746989 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:19.747092 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:19.747329 (+ 237us) server_negotiation.cc:299] Negotiation successful
0923 18:04:19.747436 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 18:04:19.751142 2663 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:04:19.754499 2663 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:04:19.770251 3137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.764454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:43875 (local address 127.0.0.1:43452)
0923 18:04:19.764681 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:19.764696 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:19.764806 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:19.765990 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:19.765995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:19.766015 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:19.766121 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.766130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.767145 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.767166 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.768555 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.768565 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.769645 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.769649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.769768 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.769809 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 18:04:19.769942 (+ 133us) client_negotiation.cc:240] Negotiation successful
0923 18:04:19.770086 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"spinlock_wait_cycles":7680,"thread_start_us":98,"threads_started":1}
I20260923 18:04:19.770258 3138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.764863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43452 (local address 127.15.139.130:43875)
0923 18:04:19.765276 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:19.765294 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:19.765315 (+ 21us) server_negotiation.cc:407] Connection header received
0923 18:04:19.765391 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:19.765396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:19.765623 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:19.765803 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:19.766262 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.767007 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.768672 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.769479 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.769674 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.769855 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:19.769974 (+ 119us) server_negotiation.cc:299] Negotiation successful
0923 18:04:19.770086 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":214,"threads_started":1}
I20260923 18:04:19.770998 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.763680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:38529 (local address 127.0.0.1:56540)
0923 18:04:19.764538 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:19.764596 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:19.764807 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:19.765642 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:19.765648 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:19.765691 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:19.765972 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.765980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.767580 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.767584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.768702 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.768710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.770054 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.770058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.770157 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.770182 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 18:04:19.770298 (+ 116us) client_negotiation.cc:240] Negotiation successful
0923 18:04:19.770839 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":55,"spinlock_wait_cycles":5248,"thread_start_us":111,"threads_started":1}
I20260923 18:04:19.771697 3134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.763834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.15.139.129:38529)
0923 18:04:19.764309 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:19.764327 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:19.764688 (+ 361us) server_negotiation.cc:407] Connection header received
0923 18:04:19.765036 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:19.765047 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:19.765275 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:19.765443 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:19.766127 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.767380 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.768825 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.769727 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.769848 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.769992 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:19.771381 (+ 1389us) server_negotiation.cc:299] Negotiation successful
0923 18:04:19.771545 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":136,"threads_started":1}
I20260923 18:04:19.771972 3136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.764176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50632 (local address 127.15.139.131:40505)
0923 18:04:19.764519 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:19.764528 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:19.764967 (+ 439us) server_negotiation.cc:407] Connection header received
0923 18:04:19.765061 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:19.765066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:19.765275 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:19.765443 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:19.766156 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.767391 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.770033 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.770991 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.771146 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.771279 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:19.771739 (+ 460us) server_negotiation.cc:299] Negotiation successful
0923 18:04:19.771843 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":119,"threads_started":1}
I20260923 18:04:19.772186 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.763974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:40505 (local address 127.0.0.1:50632)
0923 18:04:19.764681 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:19.764696 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:19.764806 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:19.765647 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:19.765654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:19.765686 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:19.765975 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.765980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.768711 (+ 2731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.768714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.769870 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.769881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.771133 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.771137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.771241 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.771267 (+ 26us) client_negotiation.cc:769] Sending connection context
0923 18:04:19.771355 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 18:04:19.771423 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":65,"spinlock_wait_cycles":5120,"thread_start_us":103,"threads_started":1}
I20260923 18:04:19.773751 2922 tablet_service.cc:1467] Processing CreateTablet for tablet 6eae182af05c4ab09a12bc1cb56a4060 (DEFAULT_TABLE table=test-table [id=57b2596e15944754b8bff6541993c74f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:19.774976 2922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6eae182af05c4ab09a12bc1cb56a4060. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:19.774809 3056 tablet_service.cc:1467] Processing CreateTablet for tablet 6eae182af05c4ab09a12bc1cb56a4060 (DEFAULT_TABLE table=test-table [id=57b2596e15944754b8bff6541993c74f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:19.775216 2788 tablet_service.cc:1467] Processing CreateTablet for tablet 6eae182af05c4ab09a12bc1cb56a4060 (DEFAULT_TABLE table=test-table [id=57b2596e15944754b8bff6541993c74f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:19.775852 3056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6eae182af05c4ab09a12bc1cb56a4060. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:19.776247 2788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6eae182af05c4ab09a12bc1cb56a4060. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:19.782254 3139 tablet_bootstrap.cc:492] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664: Bootstrap starting.
I20260923 18:04:19.784034 3140 tablet_bootstrap.cc:492] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f: Bootstrap starting.
I20260923 18:04:19.784821 3139 tablet_bootstrap.cc:654] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:19.786007 3140 tablet_bootstrap.cc:654] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:19.786175 3139 log.cc:824] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:19.787195 3140 log.cc:824] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:19.787914 3141 tablet_bootstrap.cc:492] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284: Bootstrap starting.
I20260923 18:04:19.788391 3139 tablet_bootstrap.cc:492] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664: No bootstrap required, opened a new log
I20260923 18:04:19.788626 3139 ts_tablet_manager.cc:1397] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260923 18:04:19.789428 3140 tablet_bootstrap.cc:492] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f: No bootstrap required, opened a new log
I20260923 18:04:19.789620 3140 ts_tablet_manager.cc:1397] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260923 18:04:19.790335 3141 tablet_bootstrap.cc:654] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:19.791213 3141 log.cc:824] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:19.792371 3139 raft_consensus.cc:348] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "441ed151e38b4ab1a4a185bdbd703284" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38529 } } peers { permanent_uuid: "916a62e93d2247f0824309c2ca8a9a8f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43875 } } peers { permanent_uuid: "c4ababc5f6b54151b5eff53c28deb664" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 40505 } }
I20260923 18:04:19.792654 3139 raft_consensus.cc:374] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:19.792712 3139 raft_consensus.cc:729] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ababc5f6b54151b5eff53c28deb664, State: Initialized, Role: FOLLOWER
I20260923 18:04:19.793197 3139 consensus_queue.cc:260] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "441ed151e38b4ab1a4a185bdbd703284" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38529 } } peers { permanent_uuid: "916a62e93d2247f0824309c2ca8a9a8f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43875 } } peers { permanent_uuid: "c4ababc5f6b54151b5eff53c28deb664" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 40505 } }
I20260923 18:04:19.793732 3141 tablet_bootstrap.cc:492] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284: No bootstrap required, opened a new log
I20260923 18:04:19.793931 3141 ts_tablet_manager.cc:1397] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260923 18:04:19.793792 3140 raft_consensus.cc:348] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "441ed151e38b4ab1a4a185bdbd703284" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38529 } } peers { permanent_uuid: "916a62e93d2247f0824309c2ca8a9a8f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43875 } } peers { permanent_uuid: "c4ababc5f6b54151b5eff53c28deb664" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 40505 } }
I20260923 18:04:19.794081 3140 raft_consensus.cc:374] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:19.794152 3140 raft_consensus.cc:729] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 916a62e93d2247f0824309c2ca8a9a8f, State: Initialized, Role: FOLLOWER
I20260923 18:04:19.794269 3122 heartbeater.cc:503] Master 127.15.139.190:42615 was elected leader, sending a full tablet report...
I20260923 18:04:19.794677 3140 consensus_queue.cc:260] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "441ed151e38b4ab1a4a185bdbd703284" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38529 } } peers { permanent_uuid: "916a62e93d2247f0824309c2ca8a9a8f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43875 } } peers { permanent_uuid: "c4ababc5f6b54151b5eff53c28deb664" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 40505 } }
I20260923 18:04:19.795397 2988 heartbeater.cc:503] Master 127.15.139.190:42615 was elected leader, sending a full tablet report...
I20260923 18:04:19.796012 3139 ts_tablet_manager.cc:1428] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260923 18:04:19.799098 3141 raft_consensus.cc:348] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "441ed151e38b4ab1a4a185bdbd703284" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38529 } } peers { permanent_uuid: "916a62e93d2247f0824309c2ca8a9a8f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43875 } } peers { permanent_uuid: "c4ababc5f6b54151b5eff53c28deb664" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 40505 } }
I20260923 18:04:19.799408 3141 raft_consensus.cc:374] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:19.799474 3141 raft_consensus.cc:729] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 441ed151e38b4ab1a4a185bdbd703284, State: Initialized, Role: FOLLOWER
I20260923 18:04:19.799777 3140 ts_tablet_manager.cc:1428] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f: Time spent starting tablet: real 0.010s user 0.006s sys 0.002s
I20260923 18:04:19.799978 3141 consensus_queue.cc:260] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "441ed151e38b4ab1a4a185bdbd703284" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38529 } } peers { permanent_uuid: "916a62e93d2247f0824309c2ca8a9a8f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43875 } } peers { permanent_uuid: "c4ababc5f6b54151b5eff53c28deb664" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 40505 } }
I20260923 18:04:19.800977 2854 heartbeater.cc:503] Master 127.15.139.190:42615 was elected leader, sending a full tablet report...
I20260923 18:04:19.800987 3141 ts_tablet_manager.cc:1428] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260923 18:04:19.860566 3145 raft_consensus.cc:482] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:04:19.860829 3145 raft_consensus.cc:504] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "441ed151e38b4ab1a4a185bdbd703284" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38529 } } peers { permanent_uuid: "916a62e93d2247f0824309c2ca8a9a8f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43875 } } peers { permanent_uuid: "c4ababc5f6b54151b5eff53c28deb664" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 40505 } }
I20260923 18:04:19.862166 3145 leader_election.cc:288] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 441ed151e38b4ab1a4a185bdbd703284 (127.15.139.129:38529), 916a62e93d2247f0824309c2ca8a9a8f (127.15.139.130:43875)
I20260923 18:04:19.867697 3134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.862732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:58269 (local address 127.15.139.129:38529)
0923 18:04:19.862879 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:19.862887 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:19.862908 (+ 21us) server_negotiation.cc:407] Connection header received
0923 18:04:19.862972 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:19.862977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:19.863032 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:19.863131 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:19.863454 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.864309 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.866369 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.866948 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.867094 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.867174 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:19.867433 (+ 259us) server_negotiation.cc:299] Negotiation successful
0923 18:04:19.867554 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 18:04:19.867729 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.862496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:38529 (local address 127.15.139.131:58269)
0923 18:04:19.862673 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:19.862687 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:19.862812 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:19.863163 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:19.863166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:19.863188 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:19.863334 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.863341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.864548 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.864552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.866220 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.866233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.867104 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.867108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.867262 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.867295 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 18:04:19.867422 (+ 127us) client_negotiation.cc:240] Negotiation successful
0923 18:04:19.867573 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":98}
I20260923 18:04:19.867765 3138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.863034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:53603 (local address 127.15.139.130:43875)
0923 18:04:19.863182 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:19.863187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:19.863293 (+ 106us) server_negotiation.cc:407] Connection header received
0923 18:04:19.863427 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:19.863431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:19.863486 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:19.863612 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:19.863851 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.864522 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.866416 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.867090 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.867249 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.867314 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:19.867565 (+ 251us) server_negotiation.cc:299] Negotiation successful
0923 18:04:19.867646 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 18:04:19.867739 3148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.862817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:43875 (local address 127.15.139.131:53603)
0923 18:04:19.863192 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:19.863205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:19.863300 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:19.863625 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:19.863627 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:19.863645 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:19.863740 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.863745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.864642 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.864646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.866219 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:19.866233 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.867257 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.867260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:19.867360 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.867410 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 18:04:19.867490 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 18:04:19.867587 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":125,"thread_start_us":140,"threads_started":1}
I20260923 18:04:19.868543 2808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eae182af05c4ab09a12bc1cb56a4060" candidate_uuid: "c4ababc5f6b54151b5eff53c28deb664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "441ed151e38b4ab1a4a185bdbd703284" is_pre_election: true
I20260923 18:04:19.868706 2942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eae182af05c4ab09a12bc1cb56a4060" candidate_uuid: "c4ababc5f6b54151b5eff53c28deb664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "916a62e93d2247f0824309c2ca8a9a8f" is_pre_election: true
I20260923 18:04:19.868938 2808 raft_consensus.cc:2454] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ababc5f6b54151b5eff53c28deb664 in term 0.
I20260923 18:04:19.868963 2942 raft_consensus.cc:2454] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ababc5f6b54151b5eff53c28deb664 in term 0.
I20260923 18:04:19.869509 3009 leader_election.cc:302] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 916a62e93d2247f0824309c2ca8a9a8f, c4ababc5f6b54151b5eff53c28deb664; no voters:
I20260923 18:04:19.869788 3145 raft_consensus.cc:2781] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:04:19.869896 3145 raft_consensus.cc:482] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:04:19.869930 3145 raft_consensus.cc:3037] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:19.871395 3145 raft_consensus.cc:504] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "441ed151e38b4ab1a4a185bdbd703284" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38529 } } peers { permanent_uuid: "916a62e93d2247f0824309c2ca8a9a8f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43875 } } peers { permanent_uuid: "c4ababc5f6b54151b5eff53c28deb664" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 40505 } }
I20260923 18:04:19.871999 3145 leader_election.cc:288] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [CANDIDATE]: Term 1 election: Requested vote from peers 441ed151e38b4ab1a4a185bdbd703284 (127.15.139.129:38529), 916a62e93d2247f0824309c2ca8a9a8f (127.15.139.130:43875)
I20260923 18:04:19.872607 2808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eae182af05c4ab09a12bc1cb56a4060" candidate_uuid: "c4ababc5f6b54151b5eff53c28deb664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "441ed151e38b4ab1a4a185bdbd703284"
I20260923 18:04:19.872566 2942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eae182af05c4ab09a12bc1cb56a4060" candidate_uuid: "c4ababc5f6b54151b5eff53c28deb664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "916a62e93d2247f0824309c2ca8a9a8f"
I20260923 18:04:19.872758 2942 raft_consensus.cc:3037] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:19.872758 2808 raft_consensus.cc:3037] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:19.874234 2808 raft_consensus.cc:2454] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ababc5f6b54151b5eff53c28deb664 in term 1.
I20260923 18:04:19.874236 2942 raft_consensus.cc:2454] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ababc5f6b54151b5eff53c28deb664 in term 1.
I20260923 18:04:19.874666 3009 leader_election.cc:302] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 916a62e93d2247f0824309c2ca8a9a8f, c4ababc5f6b54151b5eff53c28deb664; no voters:
I20260923 18:04:19.874938 3145 raft_consensus.cc:2781] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:19.875242 3145 raft_consensus.cc:686] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 1 LEADER]: Becoming Leader. State: Replica: c4ababc5f6b54151b5eff53c28deb664, State: Running, Role: LEADER
I20260923 18:04:19.875701 3145 consensus_queue.cc:237] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "441ed151e38b4ab1a4a185bdbd703284" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38529 } } peers { permanent_uuid: "916a62e93d2247f0824309c2ca8a9a8f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43875 } } peers { permanent_uuid: "c4ababc5f6b54151b5eff53c28deb664" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 40505 } }
I20260923 18:04:19.879127 2663 catalog_manager.cc:5495] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4ababc5f6b54151b5eff53c28deb664 (127.15.139.131). New cstate: current_term: 1 leader_uuid: "c4ababc5f6b54151b5eff53c28deb664" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "441ed151e38b4ab1a4a185bdbd703284" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "916a62e93d2247f0824309c2ca8a9a8f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ababc5f6b54151b5eff53c28deb664" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260923 18:04:19.899397 3136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.895766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50648 (local address 127.15.139.131:40505)
0923 18:04:19.895916 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:19.895920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:19.895969 (+ 49us) server_negotiation.cc:407] Connection header received
0923 18:04:19.896053 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:19.896055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:19.896113 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:19.896203 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:19.896427 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.897020 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.897902 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:19.898138 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:19.898231 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:19.898542 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:19.899026 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:19.899139 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:19.899199 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 18:04:19.899255 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 18:04:19.905792 2942 raft_consensus.cc:1261] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f [term 1 FOLLOWER]: Refusing update from remote peer c4ababc5f6b54151b5eff53c28deb664: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:19.905793 2808 raft_consensus.cc:1261] T 6eae182af05c4ab09a12bc1cb56a4060 P 441ed151e38b4ab1a4a185bdbd703284 [term 1 FOLLOWER]: Refusing update from remote peer c4ababc5f6b54151b5eff53c28deb664: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:19.906615 3145 consensus_queue.cc:1035] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [LEADER]: Connected to new peer: Peer: permanent_uuid: "441ed151e38b4ab1a4a185bdbd703284" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:19.906809 3149 consensus_queue.cc:1035] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [LEADER]: Connected to new peer: Peer: permanent_uuid: "916a62e93d2247f0824309c2ca8a9a8f" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 43875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:19.915323 3153 mvcc.cc:205] Tried to move back new op lower bound from 7332604558966231040 to 7332604558853967872. Current Snapshot: MvccSnapshot[applied={T|T < 7332604558966231040}]
I20260923 18:04:19.915532 3154 mvcc.cc:205] Tried to move back new op lower bound from 7332604558966231040 to 7332604558853967872. Current Snapshot: MvccSnapshot[applied={T|T < 7332604558966231040}]
I20260923 18:04:19.925912 2663 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:04:19.926131 2663 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:04:19.929095 2663 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acd1c1b84e4746638049bae91574a058: Sending DeleteTablet for 3 replicas of tablet 6eae182af05c4ab09a12bc1cb56a4060
I20260923 18:04:19.930092 2788 tablet_service.cc:1514] Processing DeleteTablet for tablet 6eae182af05c4ab09a12bc1cb56a4060 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:19 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:56540
I20260923 18:04:19.930092 2922 tablet_service.cc:1514] Processing DeleteTablet for tablet 6eae182af05c4ab09a12bc1cb56a4060 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:19 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:43452
I20260923 18:04:19.930792 3056 tablet_service.cc:1514] Processing DeleteTablet for tablet 6eae182af05c4ab09a12bc1cb56a4060 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:19 UTC) from {username='kudu', principal='kudu/127.15.139.190@KRBTEST.COM'} at 127.0.0.1:50632
I20260923 18:04:19.931313 3164 tablet_replica.cc:330] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664: stopping tablet replica
I20260923 18:04:19.931746 3164 raft_consensus.cc:2229] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:04:19.932046 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 2725
I20260923 18:04:19.933228 3164 raft_consensus.cc:2258] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:19.935885 3163 tablet_replica.cc:330] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f: stopping tablet replica
I20260923 18:04:19.936281 3163 raft_consensus.cc:2229] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:04:19.936659 3163 raft_consensus.cc:2258] T 6eae182af05c4ab09a12bc1cb56a4060 P 916a62e93d2247f0824309c2ca8a9a8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:19.937377 3164 ts_tablet_manager.cc:1905] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:04:19.941203 3164 ts_tablet_manager.cc:1918] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:04:19.941401 3164 log.cc:1196] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/6eae182af05c4ab09a12bc1cb56a4060
I20260923 18:04:19.941886 3164 ts_tablet_manager.cc:1939] T 6eae182af05c4ab09a12bc1cb56a4060 P c4ababc5f6b54151b5eff53c28deb664: Deleting consensus metadata
W20260923 18:04:19.942687 2647 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:38529 (error 108)
I20260923 18:04:19.942830 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 2859
I20260923 18:04:19.945561 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.945125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:38529 (local address 127.0.0.1:56542)
0923 18:04:19.945352 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:19.945440 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:38529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260923 18:04:19.945909 2647 catalog_manager.cc:4584] TS 441ed151e38b4ab1a4a185bdbd703284 (127.15.139.129:38529): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6eae182af05c4ab09a12bc1cb56a4060: Network error: Client connection negotiation failed: client connection to 127.15.139.129:38529: connect: Connection refused (error 111)
I20260923 18:04:19.948705 2649 catalog_manager.cc:4841] TS c4ababc5f6b54151b5eff53c28deb664 (127.15.139.131:40505): tablet 6eae182af05c4ab09a12bc1cb56a4060 (table test-table [id=57b2596e15944754b8bff6541993c74f]) successfully deleted
I20260923 18:04:19.950374 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 2993
I20260923 18:04:19.950609 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:19.950271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:43875 (local address 127.0.0.1:43458)
0923 18:04:19.950435 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:19.950506 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.130:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260923 18:04:19.956724 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 2631
2026-09-23T18:04:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2404 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-23T18:04:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:19Z Disabled control of system clock
I20260923 18:04:19.989657 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:35769
--webserver_interface=127.15.139.190
--webserver_port=0
--builtin_ntp_servers=127.15.139.148:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:35769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 18:04:20.105707 3169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:20.106026 3169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:20.106065 3169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:20.110724 3169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:20.110821 3169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:20.110838 3169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:20.110865 3169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:20.110881 3169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:20.116153 3169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:35769
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:35769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:20.117285 3169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:20.118680 3169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:20.128178 3175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:20.128202 3177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:20.128178 3174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:20.128582 3169 server_base.cc:1034] running on GCE node
I20260923 18:04:20.129179 3169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:20.130518 3169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:20.131786 3169 hybrid_clock.cc:648] HybridClock initialized: now 1790186660131772 us; error 44 us; skew 500 ppm
I20260923 18:04:20.133903 3169 webserver.cc:460] Webserver started at http://127.15.139.190:41763/ using document root <none> and password file <none>
I20260923 18:04:20.134557 3169 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:20.134665 3169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:20.134924 3169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:20.137274 3169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "a7cd89b2b7d44c1b8e3b455d6b50b533"
format_stamp: "Formatted at 2026-09-23 18:04:20 on dist-test-slave-vr57"
server_key: "74a53040e2ec971b0ef4fa29026760bd"
server_key_iv: "0e72db33e8c65cf167637693de7b04e0"
server_key_version: "encryptionkey@0"
I20260923 18:04:20.137884 3169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "a7cd89b2b7d44c1b8e3b455d6b50b533"
format_stamp: "Formatted at 2026-09-23 18:04:20 on dist-test-slave-vr57"
server_key: "74a53040e2ec971b0ef4fa29026760bd"
server_key_iv: "0e72db33e8c65cf167637693de7b04e0"
server_key_version: "encryptionkey@0"
I20260923 18:04:20.142331 3169 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:04:20.145196 3183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:20.146731 3169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:04:20.146844 3169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "a7cd89b2b7d44c1b8e3b455d6b50b533"
format_stamp: "Formatted at 2026-09-23 18:04:20 on dist-test-slave-vr57"
server_key: "74a53040e2ec971b0ef4fa29026760bd"
server_key_iv: "0e72db33e8c65cf167637693de7b04e0"
server_key_version: "encryptionkey@0"
I20260923 18:04:20.146942 3169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:20.158522 3169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:20.159233 3169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:20.159390 3169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:20.166684 3169 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:35769
I20260923 18:04:20.166697 3235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:35769 every 8 connection(s)
I20260923 18:04:20.167793 3169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:20.170949 3236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:20.176271 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 3169
I20260923 18:04:20.176406 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:20.176730 15918 external_mini_cluster.cc:1428] Setting key 5e8f1a6ac8c6bd3124ded003284d4a97
I20260923 18:04:20.176868 3236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533: Bootstrap starting.
I20260923 18:04:20.179471 3236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:20.180315 3236 log.cc:824] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:20.182211 3236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533: No bootstrap required, opened a new log
I20260923 18:04:20.185850 3236 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7cd89b2b7d44c1b8e3b455d6b50b533" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35769 } }
I20260923 18:04:20.186106 3236 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:20.186201 3236 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7cd89b2b7d44c1b8e3b455d6b50b533, State: Initialized, Role: FOLLOWER
I20260923 18:04:20.186424 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.178226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55912 (local address 127.15.139.190:35769)
0923 18:04:20.178905 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:20.178919 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:20.178957 (+ 38us) server_negotiation.cc:407] Connection header received
0923 18:04:20.179737 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:20.179766 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:20.180231 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:20.180575 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:20.181044 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.182001 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.182957 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.183445 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.183678 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.184544 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:20.184550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:20.184572 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:20.184986 (+ 414us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:20.185043 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:20.185063 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:20.185181 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:20.185300 (+ 119us) server_negotiation.cc:299] Negotiation successful
0923 18:04:20.185571 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":139,"threads_started":1}
I20260923 18:04:20.186722 3236 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7cd89b2b7d44c1b8e3b455d6b50b533" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35769 } }
I20260923 18:04:20.186857 3236 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:20.186942 3236 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:20.187062 3236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:20.188266 3236 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7cd89b2b7d44c1b8e3b455d6b50b533" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35769 } }
I20260923 18:04:20.188621 3236 leader_election.cc:302] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7cd89b2b7d44c1b8e3b455d6b50b533; no voters:
I20260923 18:04:20.189011 3236 leader_election.cc:288] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:20.189147 3241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:20.189390 3241 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [term 1 LEADER]: Becoming Leader. State: Replica: a7cd89b2b7d44c1b8e3b455d6b50b533, State: Running, Role: LEADER
I20260923 18:04:20.189690 3241 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7cd89b2b7d44c1b8e3b455d6b50b533" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35769 } }
I20260923 18:04:20.190317 3236 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:20.191342 3242 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7cd89b2b7d44c1b8e3b455d6b50b533" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7cd89b2b7d44c1b8e3b455d6b50b533" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35769 } } }
I20260923 18:04:20.191483 3242 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:20.191810 3243 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7cd89b2b7d44c1b8e3b455d6b50b533. Latest consensus state: current_term: 1 leader_uuid: "a7cd89b2b7d44c1b8e3b455d6b50b533" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7cd89b2b7d44c1b8e3b455d6b50b533" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 35769 } } }
I20260923 18:04:20.191905 3243 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:20.191964 3250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:20.195259 3250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:20.200539 3250 catalog_manager.cc:1347] Generated new cluster ID: 5196253e6b2e4efc8cacef127e57790e
I20260923 18:04:20.200618 3250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:20.219151 3250 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:20.220209 3250 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:20.226438 3250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533: Generated new TSK 0
I20260923 18:04:20.227211 3250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:20.236302 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.129:0
--local_ip_for_outbound_sockets=127.15.139.129
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35769
--builtin_ntp_servers=127.15.139.148:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 18:04:20.351025 3260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:20.351343 3260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:20.351414 3260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:20.355682 3260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:20.355765 3260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:20.355859 3260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.129
I20260923 18:04:20.360742 3260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.139.129
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.129
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:20.362154 3260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:20.363878 3260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:20.374503 3265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:20.374642 3260 server_base.cc:1034] running on GCE node
W20260923 18:04:20.374505 3268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:20.374505 3266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:20.375387 3260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:20.376114 3260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:20.377316 3260 hybrid_clock.cc:648] HybridClock initialized: now 1790186660377288 us; error 41 us; skew 500 ppm
I20260923 18:04:20.379240 3260 webserver.cc:460] Webserver started at http://127.15.139.129:46057/ using document root <none> and password file <none>
I20260923 18:04:20.379894 3260 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:20.380043 3260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:20.380306 3260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:20.382440 3260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/data/instance:
uuid: "471038e26b8b4643baac994967526d01"
format_stamp: "Formatted at 2026-09-23 18:04:20 on dist-test-slave-vr57"
server_key: "cf406e3b34b839ac9a679d7b8e1bba05"
server_key_iv: "35d539b7e2e8a933684482ce79bcec0c"
server_key_version: "encryptionkey@0"
I20260923 18:04:20.383062 3260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance:
uuid: "471038e26b8b4643baac994967526d01"
format_stamp: "Formatted at 2026-09-23 18:04:20 on dist-test-slave-vr57"
server_key: "cf406e3b34b839ac9a679d7b8e1bba05"
server_key_iv: "35d539b7e2e8a933684482ce79bcec0c"
server_key_version: "encryptionkey@0"
I20260923 18:04:20.387012 3260 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:04:20.389884 3274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:20.391433 3260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:04:20.391661 3260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/wal
uuid: "471038e26b8b4643baac994967526d01"
format_stamp: "Formatted at 2026-09-23 18:04:20 on dist-test-slave-vr57"
server_key: "cf406e3b34b839ac9a679d7b8e1bba05"
server_key_iv: "35d539b7e2e8a933684482ce79bcec0c"
server_key_version: "encryptionkey@0"
I20260923 18:04:20.391826 3260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:20.417116 3260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:20.417873 3260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:20.418092 3260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:20.418722 3260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:20.419808 3260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:20.419891 3260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:20.419966 3260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:20.420002 3260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:20.430321 3260 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.129:34559
I20260923 18:04:20.430342 3387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.129:34559 every 8 connection(s)
I20260923 18:04:20.431282 3260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/data/info.pb
I20260923 18:04:20.433791 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 3260
I20260923 18:04:20.433940 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-0/wal/instance
I20260923 18:04:20.434247 15918 external_mini_cluster.cc:1428] Setting key e56a44111e921386b04db751a431902f
I20260923 18:04:20.436581 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.130:0
--local_ip_for_outbound_sockets=127.15.139.130
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35769
--builtin_ntp_servers=127.15.139.148:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 18:04:20.442154 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.433227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.129:41023 (local address 127.15.139.190:35769)
0923 18:04:20.433381 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:20.433384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:20.434510 (+ 1126us) server_negotiation.cc:407] Connection header received
0923 18:04:20.436295 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:20.436300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:20.436361 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:20.436462 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:20.437390 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.438204 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.440042 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.440259 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.440337 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.441665 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:20.441668 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:20.441670 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:20.441682 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:20.441700 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:20.441708 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:20.441768 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:20.441969 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 18:04:20.442018 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 18:04:20.443493 3390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.433593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:35769 (local address 127.15.139.129:41023)
0923 18:04:20.434328 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:20.434381 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:20.436083 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:20.436620 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:20.436627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:20.436712 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:20.437247 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.437260 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.438336 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.438342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.439904 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.439919 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.440417 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.440424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.440659 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.441338 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:04:20.441342 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:20.441404 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:04:20.441497 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:20.441800 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:20.441812 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:20.441821 (+ 9us) client_negotiation.cc:769] Sending connection context
0923 18:04:20.441969 (+ 148us) client_negotiation.cc:240] Negotiation successful
0923 18:04:20.442323 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":141,"threads_started":1}
I20260923 18:04:20.444885 3388 heartbeater.cc:347] Connected to a master server at 127.15.139.190:35769
I20260923 18:04:20.445147 3388 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:20.445659 3388 heartbeater.cc:511] Master 127.15.139.190:35769 requested a full tablet report, sending...
I20260923 18:04:20.447199 3200 ts_manager.cc:194] Registered new tserver with Master: 471038e26b8b4643baac994967526d01 (127.15.139.129:34559)
I20260923 18:04:20.448365 3200 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.139.129:41023
W20260923 18:04:20.552426 3391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:20.552712 3391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:20.552785 3391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:20.556828 3391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:20.556890 3391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:20.556977 3391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.130
I20260923 18:04:20.561756 3391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.139.130
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.130
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:20.562888 3391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:20.564417 3391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:20.574404 3396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:20.574412 3399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:20.574604 3391 server_base.cc:1034] running on GCE node
W20260923 18:04:20.574502 3397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:20.575096 3391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:20.575821 3391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:20.577004 3391 hybrid_clock.cc:648] HybridClock initialized: now 1790186660576965 us; error 55 us; skew 500 ppm
I20260923 18:04:20.578874 3391 webserver.cc:460] Webserver started at http://127.15.139.130:34219/ using document root <none> and password file <none>
I20260923 18:04:20.579638 3391 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:20.579734 3391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:20.579946 3391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:20.582023 3391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/data/instance:
uuid: "071f349ff9f3420988c15652e9430984"
format_stamp: "Formatted at 2026-09-23 18:04:20 on dist-test-slave-vr57"
server_key: "ffb6e8a0dc55612926b093918ac0d66a"
server_key_iv: "baf44d74761216001d6eee2ee5f8abba"
server_key_version: "encryptionkey@0"
I20260923 18:04:20.582588 3391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance:
uuid: "071f349ff9f3420988c15652e9430984"
format_stamp: "Formatted at 2026-09-23 18:04:20 on dist-test-slave-vr57"
server_key: "ffb6e8a0dc55612926b093918ac0d66a"
server_key_iv: "baf44d74761216001d6eee2ee5f8abba"
server_key_version: "encryptionkey@0"
I20260923 18:04:20.586436 3391 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260923 18:04:20.589046 3405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:20.590340 3391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:04:20.590466 3391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/wal
uuid: "071f349ff9f3420988c15652e9430984"
format_stamp: "Formatted at 2026-09-23 18:04:20 on dist-test-slave-vr57"
server_key: "ffb6e8a0dc55612926b093918ac0d66a"
server_key_iv: "baf44d74761216001d6eee2ee5f8abba"
server_key_version: "encryptionkey@0"
I20260923 18:04:20.590574 3391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:20.605353 3391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:20.606086 3391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:20.606297 3391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:20.606931 3391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:20.608033 3391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:20.608083 3391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:20.608156 3391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:20.608193 3391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:20.616618 3391 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.130:42529
I20260923 18:04:20.616716 3518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.130:42529 every 8 connection(s)
I20260923 18:04:20.617604 3391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/data/info.pb
I20260923 18:04:20.623728 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 3391
I20260923 18:04:20.623865 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/wal/instance
I20260923 18:04:20.624189 15918 external_mini_cluster.cc:1428] Setting key d59cc28af67f4b030c9ab9bba0eafc40
I20260923 18:04:20.626106 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.139.131:0
--local_ip_for_outbound_sockets=127.15.139.131
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35769
--builtin_ntp_servers=127.15.139.148:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 18:04:20.627405 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.619483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:54769 (local address 127.15.139.190:35769)
0923 18:04:20.619653 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:20.619657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:20.620790 (+ 1133us) server_negotiation.cc:407] Connection header received
0923 18:04:20.622122 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:20.622126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:20.622182 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:20.622280 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:20.623177 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.623858 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.625222 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.625547 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.625676 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.626865 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:20.626869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:20.626872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:20.626891 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:20.626911 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:20.626920 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:20.627009 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:20.627196 (+ 187us) server_negotiation.cc:299] Negotiation successful
0923 18:04:20.627258 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 18:04:20.628389 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.619931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:35769 (local address 127.15.139.130:54769)
0923 18:04:20.620614 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:20.620655 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:20.621909 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:20.622433 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:20.622440 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:20.622524 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:20.623031 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.623043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.624028 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.624031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.625100 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.625110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.625736 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.625741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.625909 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.626607 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:04:20.626608 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:20.626649 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:04:20.626711 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:20.627010 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:20.627022 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:20.627030 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 18:04:20.627180 (+ 150us) client_negotiation.cc:240] Negotiation successful
0923 18:04:20.627584 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":111,"threads_started":1}
I20260923 18:04:20.629696 3519 heartbeater.cc:347] Connected to a master server at 127.15.139.190:35769
I20260923 18:04:20.629940 3519 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:20.630462 3519 heartbeater.cc:511] Master 127.15.139.190:35769 requested a full tablet report, sending...
I20260923 18:04:20.631656 3200 ts_manager.cc:194] Registered new tserver with Master: 071f349ff9f3420988c15652e9430984 (127.15.139.130:42529)
I20260923 18:04:20.632560 3200 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.139.130:54769
W20260923 18:04:20.738581 3522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:20.738854 3522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:20.738893 3522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:20.742987 3522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:20.743044 3522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:20.743103 3522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.139.131
I20260923 18:04:20.748015 3522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.139.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.139.131
--webserver_port=0
--tserver_master_addrs=127.15.139.190:35769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.139.131
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:20.749104 3522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:20.750468 3522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:20.760332 3530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:20.760578 3522 server_base.cc:1034] running on GCE node
W20260923 18:04:20.760332 3527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:20.760357 3528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:20.761197 3522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:20.761883 3522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:20.763085 3522 hybrid_clock.cc:648] HybridClock initialized: now 1790186660763062 us; error 59 us; skew 500 ppm
I20260923 18:04:20.765043 3522 webserver.cc:460] Webserver started at http://127.15.139.131:36145/ using document root <none> and password file <none>
I20260923 18:04:20.765679 3522 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:20.765779 3522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:20.766026 3522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:20.768167 3522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/data/instance:
uuid: "fcd934db013641b58a273ad31ab30447"
format_stamp: "Formatted at 2026-09-23 18:04:20 on dist-test-slave-vr57"
server_key: "cf242d6979cc2260424b422eff7a513a"
server_key_iv: "fd1ade0986bba68194f179a7182e1bdc"
server_key_version: "encryptionkey@0"
I20260923 18:04:20.768739 3522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance:
uuid: "fcd934db013641b58a273ad31ab30447"
format_stamp: "Formatted at 2026-09-23 18:04:20 on dist-test-slave-vr57"
server_key: "cf242d6979cc2260424b422eff7a513a"
server_key_iv: "fd1ade0986bba68194f179a7182e1bdc"
server_key_version: "encryptionkey@0"
I20260923 18:04:20.772774 3522 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:04:20.775250 3536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:20.776620 3522 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:04:20.776767 3522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/wal
uuid: "fcd934db013641b58a273ad31ab30447"
format_stamp: "Formatted at 2026-09-23 18:04:20 on dist-test-slave-vr57"
server_key: "cf242d6979cc2260424b422eff7a513a"
server_key_iv: "fd1ade0986bba68194f179a7182e1bdc"
server_key_version: "encryptionkey@0"
I20260923 18:04:20.776888 3522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:20.793018 3522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:20.793859 3522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:20.794051 3522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:20.794687 3522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 18:04:20.795871 3522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 18:04:20.795945 3522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:20.796027 3522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 18:04:20.796077 3522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:20.806277 3522 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.131:37475
I20260923 18:04:20.806348 3649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.131:37475 every 8 connection(s)
I20260923 18:04:20.807363 3522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/data/info.pb
I20260923 18:04:20.813120 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 3522
I20260923 18:04:20.813241 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/wal/instance
I20260923 18:04:20.813702 15918 external_mini_cluster.cc:1428] Setting key e50e074353e6084a68616804d5507b10
I20260923 18:04:20.817977 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.809495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.131:36761 (local address 127.15.139.190:35769)
0923 18:04:20.809698 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:20.809702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:20.810870 (+ 1168us) server_negotiation.cc:407] Connection header received
0923 18:04:20.812442 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:20.812446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:20.812502 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:20.812614 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:20.813649 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.814266 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.816184 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.816404 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.816492 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.817577 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:20.817580 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:20.817582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:20.817594 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:20.817613 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:20.817621 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:20.817681 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:20.817818 (+ 137us) server_negotiation.cc:299] Negotiation successful
0923 18:04:20.817867 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260923 18:04:20.818847 3652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.810000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:35769 (local address 127.15.139.131:36761)
0923 18:04:20.810695 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:20.810737 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:20.812217 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:20.812786 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:20.812793 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:20.812903 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:20.813499 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.813513 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.814429 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.814437 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.816003 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.816016 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.816575 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.816578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.816725 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.817331 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 18:04:20.817333 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:20.817373 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 18:04:20.817437 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:20.817696 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:20.817702 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:20.817706 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 18:04:20.817803 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 18:04:20.818044 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":137,"threads_started":1}
I20260923 18:04:20.820019 3650 heartbeater.cc:347] Connected to a master server at 127.15.139.190:35769
I20260923 18:04:20.820282 3650 heartbeater.cc:464] Registering TS with master...
I20260923 18:04:20.820832 3650 heartbeater.cc:511] Master 127.15.139.190:35769 requested a full tablet report, sending...
I20260923 18:04:20.821983 3200 ts_manager.cc:194] Registered new tserver with Master: fcd934db013641b58a273ad31ab30447 (127.15.139.131:37475)
I20260923 18:04:20.822669 3200 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.139.131:36761
I20260923 18:04:20.828313 15918 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 18:04:20.834372 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.830384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55928 (local address 127.15.139.190:35769)
0923 18:04:20.830579 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:20.830585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:20.830717 (+ 132us) server_negotiation.cc:407] Connection header received
0923 18:04:20.830877 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:20.830880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:20.830932 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:20.831020 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:20.831288 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.831901 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.832795 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.833021 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.833131 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.833884 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:20.833888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:20.833891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:20.833910 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:20.833928 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:20.833938 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:20.834029 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:20.834154 (+ 125us) server_negotiation.cc:299] Negotiation successful
0923 18:04:20.834213 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260923 18:04:20.837251 3200 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 18:04:20.840272 3200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 18:04:20.855370 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.848819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.130:42529 (local address 127.0.0.1:41054)
0923 18:04:20.849932 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:20.849946 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:20.850123 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:20.850883 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:20.850886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:20.850897 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:20.851113 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.851120 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.852535 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.852542 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.853736 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.853746 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.854801 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.854807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.854931 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.854973 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 18:04:20.855092 (+ 119us) client_negotiation.cc:240] Negotiation successful
0923 18:04:20.855209 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1048,"mutex_wait_us":192,"thread_start_us":104,"threads_started":1}
I20260923 18:04:20.855423 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.849407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41054 (local address 127.15.139.130:42529)
0923 18:04:20.849856 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:20.849868 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:20.850030 (+ 162us) server_negotiation.cc:407] Connection header received
0923 18:04:20.850336 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:20.850342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:20.850557 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:20.850697 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:20.851250 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.852408 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.853867 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.854659 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.854792 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.855007 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:20.855133 (+ 126us) server_negotiation.cc:299] Negotiation successful
0923 18:04:20.855239 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":155,"threads_started":1}
I20260923 18:04:20.856164 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.848900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50892 (local address 127.15.139.129:34559)
0923 18:04:20.850063 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:20.850075 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:20.850099 (+ 24us) server_negotiation.cc:407] Connection header received
0923 18:04:20.850373 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:20.850378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:20.850563 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:20.850706 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:20.851066 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.852326 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.854096 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.854822 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.854931 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.855104 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:20.855876 (+ 772us) server_negotiation.cc:299] Negotiation successful
0923 18:04:20.855998 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1056,"thread_start_us":151,"threads_started":1}
I20260923 18:04:20.856377 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.849085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:37475 (local address 127.0.0.1:34746)
0923 18:04:20.849996 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:20.850009 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:20.850123 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:20.850867 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:20.850871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:20.850890 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:20.851125 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.851130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.853906 (+ 2776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.853909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.854683 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.854692 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.856001 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.856004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.856099 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.856120 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 18:04:20.856188 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 18:04:20.856249 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":75,"thread_start_us":90,"threads_started":1}
I20260923 18:04:20.856734 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.848761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:34559 (local address 127.0.0.1:50892)
0923 18:04:20.849612 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:20.849666 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:20.850229 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:20.850731 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:20.850740 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:20.850769 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:20.850930 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.850938 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.852466 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.852472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.853971 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.853978 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.855167 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.855170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.855256 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.855277 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 18:04:20.855351 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 18:04:20.855418 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":85,"thread_start_us":97,"threads_started":1}
I20260923 18:04:20.856835 3666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.849593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.15.139.131:37475)
0923 18:04:20.850317 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:20.850332 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:20.850360 (+ 28us) server_negotiation.cc:407] Connection header received
0923 18:04:20.850466 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:20.850473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:20.850646 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:20.850814 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:20.852546 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.853764 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.854847 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.855662 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.855779 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.855940 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:20.856608 (+ 668us) server_negotiation.cc:299] Negotiation successful
0923 18:04:20.856685 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":134,"threads_started":1}
I20260923 18:04:20.858709 3453 tablet_service.cc:1467] Processing CreateTablet for tablet b18b3fd67ed2432cbb1860c638cc9836 (DEFAULT_TABLE table=test-table [id=660e2d4bcf85422caab030aca8c25707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:20.859861 3453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b18b3fd67ed2432cbb1860c638cc9836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:20.859444 3584 tablet_service.cc:1467] Processing CreateTablet for tablet b18b3fd67ed2432cbb1860c638cc9836 (DEFAULT_TABLE table=test-table [id=660e2d4bcf85422caab030aca8c25707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:20.859443 3322 tablet_service.cc:1467] Processing CreateTablet for tablet b18b3fd67ed2432cbb1860c638cc9836 (DEFAULT_TABLE table=test-table [id=660e2d4bcf85422caab030aca8c25707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 18:04:20.860476 3584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b18b3fd67ed2432cbb1860c638cc9836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:20.860476 3322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b18b3fd67ed2432cbb1860c638cc9836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:20.866273 3667 tablet_bootstrap.cc:492] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984: Bootstrap starting.
I20260923 18:04:20.867439 3668 tablet_bootstrap.cc:492] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01: Bootstrap starting.
I20260923 18:04:20.868709 3669 tablet_bootstrap.cc:492] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447: Bootstrap starting.
I20260923 18:04:20.868891 3667 tablet_bootstrap.cc:654] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:20.869591 3668 tablet_bootstrap.cc:654] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:20.869788 3667 log.cc:824] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:20.870430 3668 log.cc:824] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:20.870555 3669 tablet_bootstrap.cc:654] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:20.871356 3669 log.cc:824] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:20.872501 3667 tablet_bootstrap.cc:492] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984: No bootstrap required, opened a new log
I20260923 18:04:20.872493 3668 tablet_bootstrap.cc:492] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01: No bootstrap required, opened a new log
I20260923 18:04:20.872697 3668 ts_tablet_manager.cc:1397] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260923 18:04:20.872697 3667 ts_tablet_manager.cc:1397] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260923 18:04:20.873198 3669 tablet_bootstrap.cc:492] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447: No bootstrap required, opened a new log
I20260923 18:04:20.873394 3669 ts_tablet_manager.cc:1397] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260923 18:04:20.876537 3669 raft_consensus.cc:348] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471038e26b8b4643baac994967526d01" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 34559 } } peers { permanent_uuid: "071f349ff9f3420988c15652e9430984" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42529 } } peers { permanent_uuid: "fcd934db013641b58a273ad31ab30447" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 37475 } }
I20260923 18:04:20.876830 3669 raft_consensus.cc:374] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:20.876886 3669 raft_consensus.cc:729] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcd934db013641b58a273ad31ab30447, State: Initialized, Role: FOLLOWER
I20260923 18:04:20.877163 3668 raft_consensus.cc:348] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471038e26b8b4643baac994967526d01" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 34559 } } peers { permanent_uuid: "071f349ff9f3420988c15652e9430984" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42529 } } peers { permanent_uuid: "fcd934db013641b58a273ad31ab30447" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 37475 } }
I20260923 18:04:20.877163 3667 raft_consensus.cc:348] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471038e26b8b4643baac994967526d01" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 34559 } } peers { permanent_uuid: "071f349ff9f3420988c15652e9430984" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42529 } } peers { permanent_uuid: "fcd934db013641b58a273ad31ab30447" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 37475 } }
I20260923 18:04:20.877413 3668 raft_consensus.cc:374] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:20.877413 3667 raft_consensus.cc:374] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:20.877465 3668 raft_consensus.cc:729] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 471038e26b8b4643baac994967526d01, State: Initialized, Role: FOLLOWER
I20260923 18:04:20.877473 3667 raft_consensus.cc:729] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 071f349ff9f3420988c15652e9430984, State: Initialized, Role: FOLLOWER
I20260923 18:04:20.877399 3669 consensus_queue.cc:260] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471038e26b8b4643baac994967526d01" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 34559 } } peers { permanent_uuid: "071f349ff9f3420988c15652e9430984" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42529 } } peers { permanent_uuid: "fcd934db013641b58a273ad31ab30447" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 37475 } }
I20260923 18:04:20.877959 3668 consensus_queue.cc:260] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471038e26b8b4643baac994967526d01" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 34559 } } peers { permanent_uuid: "071f349ff9f3420988c15652e9430984" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42529 } } peers { permanent_uuid: "fcd934db013641b58a273ad31ab30447" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 37475 } }
I20260923 18:04:20.877947 3667 consensus_queue.cc:260] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471038e26b8b4643baac994967526d01" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 34559 } } peers { permanent_uuid: "071f349ff9f3420988c15652e9430984" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42529 } } peers { permanent_uuid: "fcd934db013641b58a273ad31ab30447" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 37475 } }
I20260923 18:04:20.878746 3669 ts_tablet_manager.cc:1428] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260923 18:04:20.878908 3668 ts_tablet_manager.cc:1428] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260923 18:04:20.879030 3650 heartbeater.cc:503] Master 127.15.139.190:35769 was elected leader, sending a full tablet report...
I20260923 18:04:20.879252 3388 heartbeater.cc:503] Master 127.15.139.190:35769 was elected leader, sending a full tablet report...
I20260923 18:04:20.879477 3667 ts_tablet_manager.cc:1428] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260923 18:04:20.879789 3519 heartbeater.cc:503] Master 127.15.139.190:35769 was elected leader, sending a full tablet report...
W20260923 18:04:20.932593 3389 tablet.cc:2369] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 18:04:20.963655 3674 raft_consensus.cc:482] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 18:04:20.963924 3674 raft_consensus.cc:504] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471038e26b8b4643baac994967526d01" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 34559 } } peers { permanent_uuid: "071f349ff9f3420988c15652e9430984" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42529 } } peers { permanent_uuid: "fcd934db013641b58a273ad31ab30447" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 37475 } }
I20260923 18:04:20.965190 3674 leader_election.cc:288] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 471038e26b8b4643baac994967526d01 (127.15.139.129:34559), fcd934db013641b58a273ad31ab30447 (127.15.139.131:37475)
I20260923 18:04:20.969638 3666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.965779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:44117 (local address 127.15.139.131:37475)
0923 18:04:20.965935 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:20.965943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:20.966133 (+ 190us) server_negotiation.cc:407] Connection header received
0923 18:04:20.966232 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:20.966234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:20.966307 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:20.966391 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:20.966734 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.967324 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.968453 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.968985 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.969134 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.969192 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:20.969417 (+ 225us) server_negotiation.cc:299] Negotiation successful
0923 18:04:20.969491 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 18:04:20.969698 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.965475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:34559 (local address 127.15.139.130:57851)
0923 18:04:20.965760 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:20.965840 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:20.966012 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:20.966317 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:20.966321 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:20.966342 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:20.966498 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.966507 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.967341 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.967346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.968324 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.968334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.969268 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.969271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.969360 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.969381 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 18:04:20.969450 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 18:04:20.969540 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":22}
I20260923 18:04:20.969864 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.965537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.131:37475 (local address 127.15.139.130:44117)
0923 18:04:20.965997 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:20.966017 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:20.966143 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:20.966456 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:20.966458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:20.966474 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 18:04:20.966608 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.966615 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.967441 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.967443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.968325 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:20.968334 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.969125 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.969129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:20.969235 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.969260 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 18:04:20.969412 (+ 152us) client_negotiation.cc:240] Negotiation successful
0923 18:04:20.969491 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":39,"thread_start_us":149,"threads_started":1}
I20260923 18:04:20.970294 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:20.965652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.139.130:57851 (local address 127.15.139.129:34559)
0923 18:04:20.965786 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:20.965794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:20.965935 (+ 141us) server_negotiation.cc:407] Connection header received
0923 18:04:20.966140 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:20.966143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:20.966209 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:20.966306 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 18:04:20.966647 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.967215 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.968493 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:20.969028 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:20.969147 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:20.969196 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:20.970083 (+ 887us) server_negotiation.cc:299] Negotiation successful
0923 18:04:20.970158 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 18:04:20.970556 3604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18b3fd67ed2432cbb1860c638cc9836" candidate_uuid: "071f349ff9f3420988c15652e9430984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcd934db013641b58a273ad31ab30447" is_pre_election: true
I20260923 18:04:20.970925 3604 raft_consensus.cc:2454] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 071f349ff9f3420988c15652e9430984 in term 0.
I20260923 18:04:20.970898 3342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18b3fd67ed2432cbb1860c638cc9836" candidate_uuid: "071f349ff9f3420988c15652e9430984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471038e26b8b4643baac994967526d01" is_pre_election: true
I20260923 18:04:20.971191 3342 raft_consensus.cc:2454] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 071f349ff9f3420988c15652e9430984 in term 0.
I20260923 18:04:20.971427 3408 leader_election.cc:302] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 071f349ff9f3420988c15652e9430984, fcd934db013641b58a273ad31ab30447; no voters:
I20260923 18:04:20.971728 3674 raft_consensus.cc:2781] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 18:04:20.971836 3674 raft_consensus.cc:482] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 18:04:20.971900 3674 raft_consensus.cc:3037] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:20.972993 3674 raft_consensus.cc:504] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471038e26b8b4643baac994967526d01" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 34559 } } peers { permanent_uuid: "071f349ff9f3420988c15652e9430984" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42529 } } peers { permanent_uuid: "fcd934db013641b58a273ad31ab30447" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 37475 } }
I20260923 18:04:20.973372 3674 leader_election.cc:288] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [CANDIDATE]: Term 1 election: Requested vote from peers 471038e26b8b4643baac994967526d01 (127.15.139.129:34559), fcd934db013641b58a273ad31ab30447 (127.15.139.131:37475)
I20260923 18:04:20.973728 3604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18b3fd67ed2432cbb1860c638cc9836" candidate_uuid: "071f349ff9f3420988c15652e9430984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcd934db013641b58a273ad31ab30447"
I20260923 18:04:20.973728 3342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18b3fd67ed2432cbb1860c638cc9836" candidate_uuid: "071f349ff9f3420988c15652e9430984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471038e26b8b4643baac994967526d01"
I20260923 18:04:20.973861 3604 raft_consensus.cc:3037] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:20.973860 3342 raft_consensus.cc:3037] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:20.975175 3604 raft_consensus.cc:2454] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 071f349ff9f3420988c15652e9430984 in term 1.
I20260923 18:04:20.975175 3342 raft_consensus.cc:2454] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 071f349ff9f3420988c15652e9430984 in term 1.
I20260923 18:04:20.975519 3407 leader_election.cc:302] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 071f349ff9f3420988c15652e9430984, 471038e26b8b4643baac994967526d01; no voters:
I20260923 18:04:20.975775 3674 raft_consensus.cc:2781] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:20.975979 3674 raft_consensus.cc:686] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 1 LEADER]: Becoming Leader. State: Replica: 071f349ff9f3420988c15652e9430984, State: Running, Role: LEADER
I20260923 18:04:20.976321 3674 consensus_queue.cc:237] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471038e26b8b4643baac994967526d01" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 34559 } } peers { permanent_uuid: "071f349ff9f3420988c15652e9430984" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42529 } } peers { permanent_uuid: "fcd934db013641b58a273ad31ab30447" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 37475 } }
I20260923 18:04:20.980147 3200 catalog_manager.cc:5495] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 reported cstate change: term changed from 0 to 1, leader changed from <none> to 071f349ff9f3420988c15652e9430984 (127.15.139.130). New cstate: current_term: 1 leader_uuid: "071f349ff9f3420988c15652e9430984" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471038e26b8b4643baac994967526d01" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 34559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "071f349ff9f3420988c15652e9430984" member_type: VOTER last_known_addr { host: "127.15.139.130" port: 42529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcd934db013641b58a273ad31ab30447" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 37475 } health_report { overall_health: UNKNOWN } } }
I20260923 18:04:21.007202 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:21.003690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41064 (local address 127.15.139.130:42529)
0923 18:04:21.003825 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:21.003832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:21.003847 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:04:21.003930 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:21.003932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:21.003992 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:21.004107 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:21.004342 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:21.004846 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:21.005730 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:21.005951 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:21.006064 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:21.006361 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:21.006794 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:21.006934 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:21.006974 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 18:04:21.007028 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 18:04:21.013422 3342 raft_consensus.cc:1261] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01 [term 1 FOLLOWER]: Refusing update from remote peer 071f349ff9f3420988c15652e9430984: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:21.013422 3604 raft_consensus.cc:1261] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447 [term 1 FOLLOWER]: Refusing update from remote peer 071f349ff9f3420988c15652e9430984: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:21.014147 3674 consensus_queue.cc:1035] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcd934db013641b58a273ad31ab30447" member_type: VOTER last_known_addr { host: "127.15.139.131" port: 37475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:21.014318 3677 consensus_queue.cc:1035] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [LEADER]: Connected to new peer: Peer: permanent_uuid: "471038e26b8b4643baac994967526d01" member_type: VOTER last_known_addr { host: "127.15.139.129" port: 34559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:21.026072 3682 mvcc.cc:205] Tried to move back new op lower bound from 7332604563502084096 to 7332604563361828864. Current Snapshot: MvccSnapshot[applied={T|T < 7332604563502084096 or (T in {7332604563502084096})}]
I20260923 18:04:21.029726 3200 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:04:21.029862 3200 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 18:04:21.032344 3200 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7cd89b2b7d44c1b8e3b455d6b50b533: Sending DeleteTablet for 3 replicas of tablet b18b3fd67ed2432cbb1860c638cc9836
I20260923 18:04:21.033128 3453 tablet_service.cc:1514] Processing DeleteTablet for tablet b18b3fd67ed2432cbb1860c638cc9836 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:21 UTC) from {username='slave'} at 127.0.0.1:41054
I20260923 18:04:21.033097 3322 tablet_service.cc:1514] Processing DeleteTablet for tablet b18b3fd67ed2432cbb1860c638cc9836 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:21 UTC) from {username='slave'} at 127.0.0.1:50892
I20260923 18:04:21.033298 3584 tablet_service.cc:1514] Processing DeleteTablet for tablet b18b3fd67ed2432cbb1860c638cc9836 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 18:04:21 UTC) from {username='slave'} at 127.0.0.1:34746
I20260923 18:04:21.033577 3689 tablet_replica.cc:330] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01: stopping tablet replica
I20260923 18:04:21.033803 3691 tablet_replica.cc:330] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447: stopping tablet replica
I20260923 18:04:21.033908 3689 raft_consensus.cc:2229] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:04:21.034183 3691 raft_consensus.cc:2229] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 18:04:21.034240 3689 raft_consensus.cc:2258] T b18b3fd67ed2432cbb1860c638cc9836 P 471038e26b8b4643baac994967526d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:21.034561 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 3260
I20260923 18:04:21.035420 3690 tablet_replica.cc:330] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984: stopping tablet replica
I20260923 18:04:21.035763 3690 raft_consensus.cc:2229] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 1 LEADER]: Raft consensus shutting down.
I20260923 18:04:21.035972 3691 raft_consensus.cc:2258] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:21.036125 3690 raft_consensus.cc:2258] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 18:04:21.037616 3691 ts_tablet_manager.cc:1905] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:04:21.037616 3690 ts_tablet_manager.cc:1905] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 18:04:21.040355 3691 ts_tablet_manager.cc:1918] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:04:21.040355 3690 ts_tablet_manager.cc:1918] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 18:04:21.040455 3690 log.cc:1196] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-1/wal/wals/b18b3fd67ed2432cbb1860c638cc9836
I20260923 18:04:21.040668 3691 log.cc:1196] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447: Deleting WAL directory at /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790186398863621-15918-0/minicluster-data/ts-2/wal/wals/b18b3fd67ed2432cbb1860c638cc9836
I20260923 18:04:21.040795 3690 ts_tablet_manager.cc:1939] T b18b3fd67ed2432cbb1860c638cc9836 P 071f349ff9f3420988c15652e9430984: Deleting consensus metadata
I20260923 18:04:21.041100 3691 ts_tablet_manager.cc:1939] T b18b3fd67ed2432cbb1860c638cc9836 P fcd934db013641b58a273ad31ab30447: Deleting consensus metadata
I20260923 18:04:21.042042 3184 catalog_manager.cc:4841] TS 071f349ff9f3420988c15652e9430984 (127.15.139.130:42529): tablet b18b3fd67ed2432cbb1860c638cc9836 (table test-table [id=660e2d4bcf85422caab030aca8c25707]) successfully deleted
I20260923 18:04:21.042167 3186 catalog_manager.cc:4841] TS fcd934db013641b58a273ad31ab30447 (127.15.139.131:37475): tablet b18b3fd67ed2432cbb1860c638cc9836 (table test-table [id=660e2d4bcf85422caab030aca8c25707]) successfully deleted
W20260923 18:04:21.044903 3185 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.139.129:34559 (error 108)
I20260923 18:04:21.045862 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:21.045632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.129:34559 (local address 127.0.0.1:50904)
0923 18:04:21.045730 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:21.045789 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.129:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 18:04:21.046025 3185 catalog_manager.cc:4584] TS 471038e26b8b4643baac994967526d01 (127.15.139.129:34559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b18b3fd67ed2432cbb1860c638cc9836: Network error: Client connection negotiation failed: client connection to 127.15.139.129:34559: connect: Connection refused (error 111)
I20260923 18:04:21.046375 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 3391
I20260923 18:04:21.052573 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 3522
I20260923 18:04:21.058396 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 3169
2026-09-23T18:04:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1099 ms)
[----------] 6 tests from EncryptionPolicyTest (10497 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:21 dist-test-slave-vr57 krb5kdc[3695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:21 dist-test-slave-vr57 krb5kdc[3695](info): set up 2 sockets
Sep 23 18:04:21 dist-test-slave-vr57 krb5kdc[3695](info): commencing operation
krb5kdc: starting...
W20260923 18:04:22.100220 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:22 dist-test-slave-vr57 krb5kdc[3695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 18:04:22.226221 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45901
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:22.341177 3708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:22.341467 3708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:22.341562 3708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:22.345587 3708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:22.345651 3708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:22.345674 3708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:22.345743 3708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:22.345780 3708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:22.351228 3708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45901
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:22.352337 3708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:22.353437 3708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:22.362564 3714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:22.362609 3716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:22.362609 3713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:22.363013 3708 server_base.cc:1034] running on GCE node
I20260923 18:04:22.363689 3708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:22.364861 3708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:22.366050 3708 hybrid_clock.cc:648] HybridClock initialized: now 1790186662366033 us; error 31 us; skew 500 ppm
Sep 23 18:04:22 dist-test-slave-vr57 krb5kdc[3695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186662, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:22.369151 3708 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 18:04:22.370235 3708 webserver.cc:460] Webserver started at http://127.0.0.1:45979/ using document root <none> and password file <none>
I20260923 18:04:22.370899 3708 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:22.371012 3708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:22.371273 3708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:22.373593 3708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "a5becd71b8c7400dad24ebf026588c17"
format_stamp: "Formatted at 2026-09-23 18:04:22 on dist-test-slave-vr57"
server_key: "809b814ff9b836b38812410dc9e32ade"
server_key_iv: "d2d59bc7507b17db1b5ea0bae896307f"
server_key_version: "encryptionkey@0"
I20260923 18:04:22.374213 3708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "a5becd71b8c7400dad24ebf026588c17"
format_stamp: "Formatted at 2026-09-23 18:04:22 on dist-test-slave-vr57"
server_key: "809b814ff9b836b38812410dc9e32ade"
server_key_iv: "d2d59bc7507b17db1b5ea0bae896307f"
server_key_version: "encryptionkey@0"
I20260923 18:04:22.378480 3708 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:04:22.381201 3723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:22.382392 3708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:04:22.382620 3708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "a5becd71b8c7400dad24ebf026588c17"
format_stamp: "Formatted at 2026-09-23 18:04:22 on dist-test-slave-vr57"
server_key: "809b814ff9b836b38812410dc9e32ade"
server_key_iv: "d2d59bc7507b17db1b5ea0bae896307f"
server_key_version: "encryptionkey@0"
I20260923 18:04:22.382745 3708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:22.404207 3708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:22.407397 3708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:22.407653 3708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:22.415050 3708 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45901
I20260923 18:04:22.415074 3775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45901 every 8 connection(s)
I20260923 18:04:22.416122 3708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:22.418977 3776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:22.422886 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 3708
I20260923 18:04:22.423027 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:22.423353 15918 external_mini_cluster.cc:1428] Setting key aab1ab65d3921c99a2386b27e3c900f4
I20260923 18:04:22.425130 3776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17: Bootstrap starting.
I20260923 18:04:22.427452 3776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:22.428253 3776 log.cc:824] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:22.430761 3776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17: No bootstrap required, opened a new log
Sep 23 18:04:22 dist-test-slave-vr57 krb5kdc[3695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 18:04:22.434928 3776 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5becd71b8c7400dad24ebf026588c17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45901 } }
I20260923 18:04:22.435222 3776 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:22.435299 3776 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5becd71b8c7400dad24ebf026588c17, State: Initialized, Role: FOLLOWER
I20260923 18:04:22.435833 3776 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5becd71b8c7400dad24ebf026588c17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45901 } }
I20260923 18:04:22.436017 3776 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:22.436092 3776 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:22.436211 3776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:22.437139 3776 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5becd71b8c7400dad24ebf026588c17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45901 } }
I20260923 18:04:22.437525 3776 leader_election.cc:302] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5becd71b8c7400dad24ebf026588c17; no voters:
I20260923 18:04:22.437917 3776 leader_election.cc:288] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:22.437994 3781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:22.438203 3781 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [term 1 LEADER]: Becoming Leader. State: Replica: a5becd71b8c7400dad24ebf026588c17, State: Running, Role: LEADER
I20260923 18:04:22.438501 3781 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5becd71b8c7400dad24ebf026588c17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45901 } }
I20260923 18:04:22.439232 3776 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:22.440248 3782 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5becd71b8c7400dad24ebf026588c17. Latest consensus state: current_term: 1 leader_uuid: "a5becd71b8c7400dad24ebf026588c17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5becd71b8c7400dad24ebf026588c17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45901 } } }
I20260923 18:04:22.440280 3783 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5becd71b8c7400dad24ebf026588c17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5becd71b8c7400dad24ebf026588c17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45901 } } }
I20260923 18:04:22.440364 3782 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:22.440393 3783 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:22.440758 3790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:22.440930 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:22.424930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36449 (local address 127.0.0.1:45901)
0923 18:04:22.425522 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:22.425540 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:22.425581 (+ 41us) server_negotiation.cc:407] Connection header received
0923 18:04:22.426236 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:22.426252 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:22.426620 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:22.426999 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:22.427959 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:22.429299 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:22.430566 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:22.432263 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:22.432424 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:22.435165 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:22.435171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:22.435190 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:22.435224 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:22.437091 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:22.437621 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:22.437627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:22.437633 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:22.437779 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:22.438147 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:22.438150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:22.438152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:22.438791 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:22.439030 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:22.439355 (+ 325us) server_negotiation.cc:299] Negotiation successful
0923 18:04:22.439640 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":165,"threads_started":1}
I20260923 18:04:22.444034 3790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:22.444519 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:22.450472 3790 catalog_manager.cc:1347] Generated new cluster ID: 9d14a06e6346420abc5c310ae15781ab
I20260923 18:04:22.450608 3790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:22.456195 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:22.446984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45559 (local address 127.0.0.1:45901)
0923 18:04:22.447093 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:22.447097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:22.447112 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:04:22.447208 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:22.447211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:22.447255 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:22.447339 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:22.448200 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:22.448826 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:22.449647 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:22.449911 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:22.450025 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:22.452575 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:22.452579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:22.452582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:22.452610 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:22.454336 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:22.454854 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:22.454858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:22.454860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:22.454947 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:22.455270 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:22.455273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:22.455275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:22.455495 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:22.455669 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:22.455933 (+ 264us) server_negotiation.cc:299] Negotiation successful
0923 18:04:22.456033 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 18:04:22.466907 3790 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:22.467971 3790 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:22.479691 3790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5becd71b8c7400dad24ebf026588c17: Generated new TSK 0
I20260923 18:04:22.480412 3790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:22.485190 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 3708
W20260923 18:04:22.491000 3693 connection.cc:535] client connection to 127.0.0.1:45901 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45901): Connection reset by peer (error 104)
2026-09-23T18:04:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:22 dist-test-slave-vr57 krb5kdc[3807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:22 dist-test-slave-vr57 krb5kdc[3807](info): set up 2 sockets
Sep 23 18:04:22 dist-test-slave-vr57 krb5kdc[3807](info): commencing operation
krb5kdc: starting...
W20260923 18:04:23.580410 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:23 dist-test-slave-vr57 krb5kdc[3807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 18:04:23.710043 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43857
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:23.824246 3820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:23.824594 3820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:23.824671 3820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:23.828918 3820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:23.829043 3820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:23.829118 3820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:23.829155 3820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:23.829187 3820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:23.834750 3820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43857
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:23.835979 3820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:23.837067 3820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:23.846045 3825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:23.846053 3826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:23.846197 3820 server_base.cc:1034] running on GCE node
W20260923 18:04:23.846411 3828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:23.847023 3820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:23.848351 3820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:23.849579 3820 hybrid_clock.cc:648] HybridClock initialized: now 1790186663849525 us; error 69 us; skew 500 ppm
Sep 23 18:04:23 dist-test-slave-vr57 krb5kdc[3807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:23.852684 3820 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 18:04:23.853657 3820 webserver.cc:460] Webserver started at http://127.0.0.1:33969/ using document root <none> and password file <none>
I20260923 18:04:23.854532 3820 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:23.854667 3820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:23.854924 3820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:23.857314 3820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "36894014066c4f19ac0cc982f94be061"
format_stamp: "Formatted at 2026-09-23 18:04:23 on dist-test-slave-vr57"
server_key: "ddb0667e24ac31f75ea9fae4d8b0e15f"
server_key_iv: "286fe15fd5765f748e66adfd8fa64f5e"
server_key_version: "encryptionkey@0"
I20260923 18:04:23.857933 3820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "36894014066c4f19ac0cc982f94be061"
format_stamp: "Formatted at 2026-09-23 18:04:23 on dist-test-slave-vr57"
server_key: "ddb0667e24ac31f75ea9fae4d8b0e15f"
server_key_iv: "286fe15fd5765f748e66adfd8fa64f5e"
server_key_version: "encryptionkey@0"
I20260923 18:04:23.861994 3820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:04:23.864689 3835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:23.866021 3820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:23.866185 3820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "36894014066c4f19ac0cc982f94be061"
format_stamp: "Formatted at 2026-09-23 18:04:23 on dist-test-slave-vr57"
server_key: "ddb0667e24ac31f75ea9fae4d8b0e15f"
server_key_iv: "286fe15fd5765f748e66adfd8fa64f5e"
server_key_version: "encryptionkey@0"
I20260923 18:04:23.866310 3820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:23.889758 3820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:23.893438 3820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:23.893679 3820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:23.901553 3820 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43857
I20260923 18:04:23.901587 3887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43857 every 8 connection(s)
I20260923 18:04:23.902560 3820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:23.905618 3888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:23.907099 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 3820
I20260923 18:04:23.907248 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:23.907604 15918 external_mini_cluster.cc:1428] Setting key f79a4c540e861bdd7483d0cef29acb75
I20260923 18:04:23.912001 3888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061: Bootstrap starting.
I20260923 18:04:23.914318 3888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:23.915087 3888 log.cc:824] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:23.916860 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:23.909077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53919 (local address 127.0.0.1:43857)
0923 18:04:23.909546 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:23.909561 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:23.909628 (+ 67us) server_negotiation.cc:407] Connection header received
0923 18:04:23.910276 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:23.910294 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:23.910660 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:23.911038 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:23.911476 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:23.912596 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:23.913524 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:23.914021 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:23.914203 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:23.915120 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:23.915125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:23.915138 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:23.915630 (+ 492us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:23.915690 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:23.915714 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:23.915855 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:23.916009 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 18:04:23.916156 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":144,"threads_started":1}
I20260923 18:04:23.917188 3888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061: No bootstrap required, opened a new log
I20260923 18:04:23.919777 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:23.921159 3888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36894014066c4f19ac0cc982f94be061" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43857 } }
I20260923 18:04:23.921420 3888 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:23.921509 3888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36894014066c4f19ac0cc982f94be061, State: Initialized, Role: FOLLOWER
I20260923 18:04:23.922037 3888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36894014066c4f19ac0cc982f94be061" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43857 } }
I20260923 18:04:23.922219 3888 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:23.922312 3888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:23.922420 3888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:23.923667 3888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36894014066c4f19ac0cc982f94be061" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43857 } }
I20260923 18:04:23.924088 3888 leader_election.cc:302] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36894014066c4f19ac0cc982f94be061; no voters:
I20260923 18:04:23.924425 3888 leader_election.cc:288] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:23.924563 3898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:23.924808 3898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [term 1 LEADER]: Becoming Leader. State: Replica: 36894014066c4f19ac0cc982f94be061, State: Running, Role: LEADER
I20260923 18:04:23.925136 3898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36894014066c4f19ac0cc982f94be061" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43857 } }
I20260923 18:04:23.925911 3888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:23.926775 3900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36894014066c4f19ac0cc982f94be061. Latest consensus state: current_term: 1 leader_uuid: "36894014066c4f19ac0cc982f94be061" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36894014066c4f19ac0cc982f94be061" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43857 } } }
I20260923 18:04:23.926926 3900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:23.927279 3899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36894014066c4f19ac0cc982f94be061" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36894014066c4f19ac0cc982f94be061" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43857 } } }
I20260923 18:04:23.927322 3907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:23.927380 3899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:23.927316 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:23.921828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35955 (local address 127.0.0.1:43857)
0923 18:04:23.921939 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:23.921946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:23.922155 (+ 209us) server_negotiation.cc:407] Connection header received
0923 18:04:23.922309 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:23.922313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:23.922370 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:23.922605 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:23.922840 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:23.923627 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:23.924736 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:23.925086 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:23.925193 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:23.926460 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:23.926464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:23.926467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:23.926480 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:23.926513 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:23.926526 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:23.926615 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:23.927102 (+ 487us) server_negotiation.cc:299] Negotiation successful
0923 18:04:23.927138 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":39}
I20260923 18:04:23.930462 3907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:23.936194 3907 catalog_manager.cc:1347] Generated new cluster ID: f10579dd4de0485f8b940e38f7250d35
I20260923 18:04:23.936277 3907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:23.950872 3907 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:23.952220 3907 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:23.964442 3907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36894014066c4f19ac0cc982f94be061: Generated new TSK 0
I20260923 18:04:23.965251 3907 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:23.973167 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 3820
W20260923 18:04:23.979449 3805 connection.cc:535] client connection to 127.0.0.1:43857 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43857): Connection reset by peer (error 104)
2026-09-23T18:04:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:24 dist-test-slave-vr57 krb5kdc[3919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:24 dist-test-slave-vr57 krb5kdc[3919](info): set up 2 sockets
Sep 23 18:04:24 dist-test-slave-vr57 krb5kdc[3919](info): commencing operation
krb5kdc: starting...
W20260923 18:04:25.024142 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:25 dist-test-slave-vr57 krb5kdc[3919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 18:04:25.147308 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38327
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:25.259690 3932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:25.260039 3932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:25.260125 3932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:25.264292 3932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:25.264384 3932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:25.264420 3932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:25.264482 3932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:25.264519 3932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:25.269850 3932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38327
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:25.270946 3932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:25.272070 3932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:25.280670 3938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:25.280671 3937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:25.280671 3940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:25.280903 3932 server_base.cc:1034] running on GCE node
I20260923 18:04:25.281600 3932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:25.282928 3932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:25.284165 3932 hybrid_clock.cc:648] HybridClock initialized: now 1790186665284152 us; error 38 us; skew 500 ppm
Sep 23 18:04:25 dist-test-slave-vr57 krb5kdc[3919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186665, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:25.294798 3932 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 18:04:25.295996 3932 webserver.cc:460] Webserver started at http://127.0.0.1:35707/ using document root <none> and password file <none>
I20260923 18:04:25.296635 3932 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:25.296736 3932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:25.296998 3932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:25.299237 3932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "ed50350d29f04f3db0e3f30a4cc0754d"
format_stamp: "Formatted at 2026-09-23 18:04:25 on dist-test-slave-vr57"
server_key: "eb12441254d596356117899c73ccb260"
server_key_iv: "c59f95098ef7ab4c3825e9c2186aedc7"
server_key_version: "encryptionkey@0"
I20260923 18:04:25.299857 3932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "ed50350d29f04f3db0e3f30a4cc0754d"
format_stamp: "Formatted at 2026-09-23 18:04:25 on dist-test-slave-vr57"
server_key: "eb12441254d596356117899c73ccb260"
server_key_iv: "c59f95098ef7ab4c3825e9c2186aedc7"
server_key_version: "encryptionkey@0"
I20260923 18:04:25.303974 3932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260923 18:04:25.307257 3947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:25.308898 3932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:25.309080 3932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "ed50350d29f04f3db0e3f30a4cc0754d"
format_stamp: "Formatted at 2026-09-23 18:04:25 on dist-test-slave-vr57"
server_key: "eb12441254d596356117899c73ccb260"
server_key_iv: "c59f95098ef7ab4c3825e9c2186aedc7"
server_key_version: "encryptionkey@0"
I20260923 18:04:25.309230 3932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:25.314320 3932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:25.317612 3932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:25.317833 3932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:25.325326 3932 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38327
I20260923 18:04:25.325350 3999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38327 every 8 connection(s)
I20260923 18:04:25.326345 3932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:25.329336 4000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:25.334158 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 3932
I20260923 18:04:25.334304 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:25.334599 15918 external_mini_cluster.cc:1428] Setting key c1386e387effbc1f4b3da3b659e6984a
I20260923 18:04:25.336170 4000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d: Bootstrap starting.
I20260923 18:04:25.338860 4000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:25.339902 4000 log.cc:824] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:25.340642 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:25.336226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39509 (local address 127.0.0.1:38327)
0923 18:04:25.336738 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:25.336754 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:25.336791 (+ 37us) server_negotiation.cc:407] Connection header received
0923 18:04:25.337520 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:25.337540 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:25.337859 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:25.338219 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:25.339157 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:25.339164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:25.339184 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:25.339201 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:25.339279 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:25.339306 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:25.339426 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:25.339543 (+ 117us) server_negotiation.cc:299] Negotiation successful
0923 18:04:25.339713 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":158,"threads_started":1}
I20260923 18:04:25.342334 4000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d: No bootstrap required, opened a new log
I20260923 18:04:25.343659 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:25.347481 4000 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed50350d29f04f3db0e3f30a4cc0754d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38327 } }
I20260923 18:04:25.347792 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:25.346008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51757 (local address 127.0.0.1:38327)
0923 18:04:25.346189 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:25.346193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:25.346408 (+ 215us) server_negotiation.cc:407] Connection header received
0923 18:04:25.346489 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:25.346492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:25.346531 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:25.346632 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:25.347341 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:25.347345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:25.347347 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:25.347358 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:25.347386 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:25.347399 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:25.347483 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:25.347635 (+ 152us) server_negotiation.cc:299] Negotiation successful
0923 18:04:25.347679 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 18:04:25.347815 4000 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:25.347865 4000 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed50350d29f04f3db0e3f30a4cc0754d, State: Initialized, Role: FOLLOWER
I20260923 18:04:25.348352 4000 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed50350d29f04f3db0e3f30a4cc0754d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38327 } }
I20260923 18:04:25.348480 4000 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:25.348567 4000 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:25.348673 4000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:25.349774 4000 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed50350d29f04f3db0e3f30a4cc0754d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38327 } }
I20260923 18:04:25.350098 4000 leader_election.cc:302] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed50350d29f04f3db0e3f30a4cc0754d; no voters:
I20260923 18:04:25.350361 4000 leader_election.cc:288] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:25.350488 4010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:25.350760 4010 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [term 1 LEADER]: Becoming Leader. State: Replica: ed50350d29f04f3db0e3f30a4cc0754d, State: Running, Role: LEADER
I20260923 18:04:25.351135 4010 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed50350d29f04f3db0e3f30a4cc0754d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38327 } }
I20260923 18:04:25.351729 4000 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:25.352933 4011 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed50350d29f04f3db0e3f30a4cc0754d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed50350d29f04f3db0e3f30a4cc0754d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38327 } } }
I20260923 18:04:25.353062 4011 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:25.353669 4012 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed50350d29f04f3db0e3f30a4cc0754d. Latest consensus state: current_term: 1 leader_uuid: "ed50350d29f04f3db0e3f30a4cc0754d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed50350d29f04f3db0e3f30a4cc0754d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38327 } } }
I20260923 18:04:25.353883 4012 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:25.353833 4019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:25.357092 4019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:25.363180 4019 catalog_manager.cc:1347] Generated new cluster ID: 5262adb6fdff41888e8012877d484387
I20260923 18:04:25.363260 4019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:25.376677 4019 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:25.377051 4019 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:25.389964 4019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed50350d29f04f3db0e3f30a4cc0754d: Generated new TSK 0
I20260923 18:04:25.390794 4019 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:25.396435 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 3932
2026-09-23T18:04:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:25 dist-test-slave-vr57 krb5kdc[4031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:25 dist-test-slave-vr57 krb5kdc[4031](info): set up 2 sockets
Sep 23 18:04:25 dist-test-slave-vr57 krb5kdc[4031](info): commencing operation
krb5kdc: starting...
W20260923 18:04:26.444635 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:26 dist-test-slave-vr57 krb5kdc[4031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 18:04:26.572464 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:26.681664 4044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:26.681932 4044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:26.681972 4044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:26.686031 4044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:26.686100 4044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:26.686120 4044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:26.686136 4044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:26.686182 4044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:26.691459 4044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:26.692545 4044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:26.693646 4044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:26.702171 4050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:26.702131 4049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:26.702152 4052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:26.702533 4044 server_base.cc:1034] running on GCE node
I20260923 18:04:26.703253 4044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:26.704571 4044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:26.705761 4044 hybrid_clock.cc:648] HybridClock initialized: now 1790186666705748 us; error 28 us; skew 500 ppm
Sep 23 18:04:26 dist-test-slave-vr57 krb5kdc[4031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:26.708747 4044 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 18:04:26.709794 4044 webserver.cc:460] Webserver started at http://127.0.0.1:36787/ using document root <none> and password file <none>
I20260923 18:04:26.710376 4044 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:26.710472 4044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:26.710783 4044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:26.712934 4044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "cc5559d57fcd4c4fa7fe3e35f0579b5a"
format_stamp: "Formatted at 2026-09-23 18:04:26 on dist-test-slave-vr57"
server_key: "5bcff61bf3d0cf4e49de78cf8eb715e6"
server_key_iv: "56510a28946742d94a5bc60cb796c878"
server_key_version: "encryptionkey@0"
I20260923 18:04:26.713512 4044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "cc5559d57fcd4c4fa7fe3e35f0579b5a"
format_stamp: "Formatted at 2026-09-23 18:04:26 on dist-test-slave-vr57"
server_key: "5bcff61bf3d0cf4e49de78cf8eb715e6"
server_key_iv: "56510a28946742d94a5bc60cb796c878"
server_key_version: "encryptionkey@0"
I20260923 18:04:26.717232 4044 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260923 18:04:26.719699 4059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:26.720902 4044 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260923 18:04:26.721042 4044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "cc5559d57fcd4c4fa7fe3e35f0579b5a"
format_stamp: "Formatted at 2026-09-23 18:04:26 on dist-test-slave-vr57"
server_key: "5bcff61bf3d0cf4e49de78cf8eb715e6"
server_key_iv: "56510a28946742d94a5bc60cb796c878"
server_key_version: "encryptionkey@0"
I20260923 18:04:26.721163 4044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:26.738780 4044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:26.742374 4044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:26.742606 4044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:26.750044 4044 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35759
I20260923 18:04:26.750119 4111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35759 every 8 connection(s)
I20260923 18:04:26.751073 4044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:26.753949 4112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:26.759140 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 4044
I20260923 18:04:26.759312 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:26.759406 4112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a: Bootstrap starting.
I20260923 18:04:26.759717 15918 external_mini_cluster.cc:1428] Setting key 71e5dc31d9fae56463f452e5a49d3fcc
I20260923 18:04:26.761860 4112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:26.762585 4112 log.cc:824] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:26.764703 4112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a: No bootstrap required, opened a new log
Sep 23 18:04:26 dist-test-slave-vr57 krb5kdc[4031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 18:04:26.769320 4112 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5559d57fcd4c4fa7fe3e35f0579b5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35759 } }
I20260923 18:04:26.769603 4112 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:26.769656 4112 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc5559d57fcd4c4fa7fe3e35f0579b5a, State: Initialized, Role: FOLLOWER
I20260923 18:04:26.770149 4112 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5559d57fcd4c4fa7fe3e35f0579b5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35759 } }
I20260923 18:04:26.770346 4112 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:26.770427 4112 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:26.770540 4112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:26.771862 4112 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5559d57fcd4c4fa7fe3e35f0579b5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35759 } }
I20260923 18:04:26.772238 4112 leader_election.cc:302] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc5559d57fcd4c4fa7fe3e35f0579b5a; no voters:
I20260923 18:04:26.772631 4112 leader_election.cc:288] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:26.772725 4117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:26.772956 4117 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [term 1 LEADER]: Becoming Leader. State: Replica: cc5559d57fcd4c4fa7fe3e35f0579b5a, State: Running, Role: LEADER
I20260923 18:04:26.773343 4117 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5559d57fcd4c4fa7fe3e35f0579b5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35759 } }
I20260923 18:04:26.774008 4112 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:26.775117 4119 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc5559d57fcd4c4fa7fe3e35f0579b5a. Latest consensus state: current_term: 1 leader_uuid: "cc5559d57fcd4c4fa7fe3e35f0579b5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5559d57fcd4c4fa7fe3e35f0579b5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35759 } } }
I20260923 18:04:26.775246 4119 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:26.775646 4118 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc5559d57fcd4c4fa7fe3e35f0579b5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5559d57fcd4c4fa7fe3e35f0579b5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35759 } } }
I20260923 18:04:26.775781 4118 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:26.775856 4115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:26.761099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52229 (local address 127.0.0.1:35759)
0923 18:04:26.761524 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:26.761540 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:26.761576 (+ 36us) server_negotiation.cc:407] Connection header received
0923 18:04:26.762264 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:26.762283 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:26.762630 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:26.762958 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:26.763827 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:26.764858 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:26.766034 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:26.766511 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:26.766635 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:26.769635 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:26.769641 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:26.769658 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:26.769700 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:26.771863 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:26.772343 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:26.772352 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:26.772359 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:26.772507 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:26.772865 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:26.772872 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:26.772876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:26.773412 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:26.773605 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:26.774307 (+ 702us) server_negotiation.cc:299] Negotiation successful
0923 18:04:26.774652 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":129,"threads_started":1}
I20260923 18:04:26.779150 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260923 18:04:26.779261 4132 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:04:26.779320 4132 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:04:26.779398 4126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:26.780172 4126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:26.785833 4126 catalog_manager.cc:1347] Generated new cluster ID: c5adc5cf4e3b4d478e700b477689f23d
I20260923 18:04:26.785897 4126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:26.790529 4115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:26.782181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49135 (local address 127.0.0.1:35759)
0923 18:04:26.782289 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:26.782293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:26.782525 (+ 232us) server_negotiation.cc:407] Connection header received
0923 18:04:26.782689 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:26.782697 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:26.782736 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:26.782820 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:26.783483 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:26.784152 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:26.785447 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:26.785784 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:26.785884 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:26.787377 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:26.787380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:26.787385 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:26.787411 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:26.788906 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:26.789330 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:26.789334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:26.789336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:26.789415 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:26.789692 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:26.789696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:26.789698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:26.789864 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:26.789999 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:26.790234 (+ 235us) server_negotiation.cc:299] Negotiation successful
0923 18:04:26.790350 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 18:04:26.799127 4126 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:26.800271 4126 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:26.808027 4126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc5559d57fcd4c4fa7fe3e35f0579b5a: Generated new TSK 0
I20260923 18:04:26.808741 4126 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:26.817380 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 4044
2026-09-23T18:04:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:26 dist-test-slave-vr57 krb5kdc[4143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:26 dist-test-slave-vr57 krb5kdc[4143](info): set up 2 sockets
Sep 23 18:04:26 dist-test-slave-vr57 krb5kdc[4143](info): commencing operation
krb5kdc: starting...
W20260923 18:04:27.880190 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:27 dist-test-slave-vr57 krb5kdc[4143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 18:04:28.006580 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:28.120402 4156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:28.120685 4156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:28.120728 4156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:28.124871 4156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:28.124940 4156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:28.124962 4156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:28.124979 4156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:28.125020 4156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:28.130295 4156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:28.131408 4156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:28.132550 4156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:28.141178 4161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:28.141170 4162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:28.141170 4164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:28.141222 4156 server_base.cc:1034] running on GCE node
I20260923 18:04:28.142227 4156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:28.143462 4156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:28.144711 4156 hybrid_clock.cc:648] HybridClock initialized: now 1790186668144697 us; error 37 us; skew 500 ppm
Sep 23 18:04:28 dist-test-slave-vr57 krb5kdc[4143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186668, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:28.147655 4156 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 18:04:28.148588 4156 webserver.cc:460] Webserver started at http://127.0.0.1:44951/ using document root <none> and password file <none>
I20260923 18:04:28.149183 4156 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:28.149278 4156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:28.149521 4156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:28.151657 4156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "793b4f24201642f1b0c3db054e192aac"
format_stamp: "Formatted at 2026-09-23 18:04:28 on dist-test-slave-vr57"
server_key: "cfc2485296f0b4d5b2ef7a9a43422e2c"
server_key_iv: "e1dde52dc63e3690fce757c08d697041"
server_key_version: "encryptionkey@0"
I20260923 18:04:28.152230 4156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "793b4f24201642f1b0c3db054e192aac"
format_stamp: "Formatted at 2026-09-23 18:04:28 on dist-test-slave-vr57"
server_key: "cfc2485296f0b4d5b2ef7a9a43422e2c"
server_key_iv: "e1dde52dc63e3690fce757c08d697041"
server_key_version: "encryptionkey@0"
I20260923 18:04:28.156026 4156 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:04:28.158605 4171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:28.159888 4156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:04:28.160022 4156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "793b4f24201642f1b0c3db054e192aac"
format_stamp: "Formatted at 2026-09-23 18:04:28 on dist-test-slave-vr57"
server_key: "cfc2485296f0b4d5b2ef7a9a43422e2c"
server_key_iv: "e1dde52dc63e3690fce757c08d697041"
server_key_version: "encryptionkey@0"
I20260923 18:04:28.160115 4156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:28.170655 4156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:28.173770 4156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:28.173983 4156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:28.181398 4156 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44611
I20260923 18:04:28.181423 4223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44611 every 8 connection(s)
I20260923 18:04:28.182389 4156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:28.183122 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 4156
I20260923 18:04:28.183262 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:28.183614 15918 external_mini_cluster.cc:1428] Setting key e5e86278bcda9eff98c550b069680406
I20260923 18:04:28.185822 4224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:28.191474 4224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac: Bootstrap starting.
I20260923 18:04:28.192551 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:28.185028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55729 (local address 127.0.0.1:44611)
0923 18:04:28.185468 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:28.185483 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:28.185518 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:04:28.186268 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:28.186286 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:28.186640 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:28.186983 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:28.187419 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:28.188488 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:28.189397 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:28.189869 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:28.190044 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:28.190934 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:28.190939 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:28.190951 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:28.191330 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:28.191382 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:28.191404 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:28.191516 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:28.191670 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 18:04:28.191873 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":153,"threads_started":1}
I20260923 18:04:28.194072 4224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:28.194976 4224 log.cc:824] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:28.195441 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:28.197359 4224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac: No bootstrap required, opened a new log
I20260923 18:04:28.201148 4224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b4f24201642f1b0c3db054e192aac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260923 18:04:28.201385 4224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:28.201529 4224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 793b4f24201642f1b0c3db054e192aac, State: Initialized, Role: FOLLOWER
I20260923 18:04:28.201810 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:28.197703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34747 (local address 127.0.0.1:44611)
0923 18:04:28.197827 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:28.197831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:28.198052 (+ 221us) server_negotiation.cc:407] Connection header received
0923 18:04:28.198242 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:28.198244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:28.198282 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:28.198365 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:28.198630 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:28.199139 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:28.200048 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:28.200266 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:28.200377 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:28.201203 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:28.201209 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:28.201216 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:28.201227 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:28.201247 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:28.201257 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:28.201336 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:28.201480 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 18:04:28.201529 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 18:04:28.202054 4224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b4f24201642f1b0c3db054e192aac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260923 18:04:28.202226 4224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:28.202316 4224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:28.202416 4224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:28.203334 4224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b4f24201642f1b0c3db054e192aac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260923 18:04:28.203706 4224 leader_election.cc:302] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 793b4f24201642f1b0c3db054e192aac; no voters:
I20260923 18:04:28.204044 4224 leader_election.cc:288] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:28.204178 4234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:28.204401 4234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [term 1 LEADER]: Becoming Leader. State: Replica: 793b4f24201642f1b0c3db054e192aac, State: Running, Role: LEADER
I20260923 18:04:28.204738 4234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b4f24201642f1b0c3db054e192aac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260923 18:04:28.205268 4224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:28.206434 4235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "793b4f24201642f1b0c3db054e192aac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b4f24201642f1b0c3db054e192aac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } } }
I20260923 18:04:28.206415 4236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 793b4f24201642f1b0c3db054e192aac. Latest consensus state: current_term: 1 leader_uuid: "793b4f24201642f1b0c3db054e192aac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b4f24201642f1b0c3db054e192aac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } } }
I20260923 18:04:28.206562 4235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:28.206576 4236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:28.206974 4243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:28.209756 4243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:28.215714 4243 catalog_manager.cc:1347] Generated new cluster ID: a21e7850007649828dad298b8974275d
I20260923 18:04:28.215787 4243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:28.230370 4243 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:28.231691 4243 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:28.240787 4243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 793b4f24201642f1b0c3db054e192aac: Generated new TSK 0
I20260923 18:04:28.241667 4243 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:28.248812 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 4156
2026-09-23T18:04:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:28 dist-test-slave-vr57 krb5kdc[4255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:28 dist-test-slave-vr57 krb5kdc[4255](info): set up 2 sockets
Sep 23 18:04:28 dist-test-slave-vr57 krb5kdc[4255](info): commencing operation
krb5kdc: starting...
W20260923 18:04:29.320991 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:29 dist-test-slave-vr57 krb5kdc[4255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 18:04:29.446755 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37873
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:29.564774 4268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:29.565145 4268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:29.565253 4268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:29.569538 4268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:29.569638 4268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:29.569677 4268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:29.569737 4268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:29.569779 4268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:29.575178 4268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:29.576762 4268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:29.577850 4268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:29.586777 4273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:29.586757 4274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:29.586760 4276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:29.587081 4268 server_base.cc:1034] running on GCE node
I20260923 18:04:29.587815 4268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:29.588965 4268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:29.590178 4268 hybrid_clock.cc:648] HybridClock initialized: now 1790186669590167 us; error 28 us; skew 500 ppm
Sep 23 18:04:29 dist-test-slave-vr57 krb5kdc[4255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186669, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:29.593034 4268 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 18:04:29.594022 4268 webserver.cc:460] Webserver started at http://127.0.0.1:39145/ using document root <none> and password file <none>
I20260923 18:04:29.594619 4268 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:29.594719 4268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:29.594957 4268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:29.597082 4268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "c644028baf4a488badd04e26b5048286"
format_stamp: "Formatted at 2026-09-23 18:04:29 on dist-test-slave-vr57"
server_key: "2e7aafc82b8f58665a236f91d36b3831"
server_key_iv: "051585b3299a08a4f6c2d88029a6b801"
server_key_version: "encryptionkey@0"
I20260923 18:04:29.597674 4268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "c644028baf4a488badd04e26b5048286"
format_stamp: "Formatted at 2026-09-23 18:04:29 on dist-test-slave-vr57"
server_key: "2e7aafc82b8f58665a236f91d36b3831"
server_key_iv: "051585b3299a08a4f6c2d88029a6b801"
server_key_version: "encryptionkey@0"
I20260923 18:04:29.601670 4268 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260923 18:04:29.604346 4283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:29.605732 4268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:29.605885 4268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "c644028baf4a488badd04e26b5048286"
format_stamp: "Formatted at 2026-09-23 18:04:29 on dist-test-slave-vr57"
server_key: "2e7aafc82b8f58665a236f91d36b3831"
server_key_iv: "051585b3299a08a4f6c2d88029a6b801"
server_key_version: "encryptionkey@0"
I20260923 18:04:29.606026 4268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:29.610890 4268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:29.613871 4268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:29.614073 4268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:29.621466 4268 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37873
I20260923 18:04:29.621491 4335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37873 every 8 connection(s)
I20260923 18:04:29.622483 4268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:29.623257 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 4268
I20260923 18:04:29.623395 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:29.623759 15918 external_mini_cluster.cc:1428] Setting key 045085e201a5724c700945bbf941121b
I20260923 18:04:29.626116 4336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:29.629462 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:29.625298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58921 (local address 127.0.0.1:37873)
0923 18:04:29.625792 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:29.625807 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:29.625853 (+ 46us) server_negotiation.cc:407] Connection header received
0923 18:04:29.626616 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:29.626637 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:29.626964 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:29.627364 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:29.628257 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:29.628262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:29.628274 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:29.628283 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:29.628330 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:29.628349 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:29.628517 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:29.628588 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 18:04:29.628754 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":160,"threads_started":1}
I20260923 18:04:29.632649 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:29.632875 4336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286: Bootstrap starting.
I20260923 18:04:29.635327 4336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:29.636341 4336 log.cc:824] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:29.636494 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:29.634778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50047 (local address 127.0.0.1:37873)
0923 18:04:29.634908 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:29.634912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:29.635054 (+ 142us) server_negotiation.cc:407] Connection header received
0923 18:04:29.635141 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:29.635146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:29.635191 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:29.635285 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:29.636096 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:29.636100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:29.636104 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:29.636119 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:29.636149 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:29.636161 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:29.636256 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:29.636330 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 18:04:29.636368 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 18:04:29.638432 4336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286: No bootstrap required, opened a new log
I20260923 18:04:29.642181 4336 raft_consensus.cc:348] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c644028baf4a488badd04e26b5048286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } }
I20260923 18:04:29.642401 4336 raft_consensus.cc:374] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:29.642436 4336 raft_consensus.cc:729] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c644028baf4a488badd04e26b5048286, State: Initialized, Role: FOLLOWER
I20260923 18:04:29.642788 4336 consensus_queue.cc:260] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c644028baf4a488badd04e26b5048286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } }
I20260923 18:04:29.642897 4336 raft_consensus.cc:388] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:29.642935 4336 raft_consensus.cc:482] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:29.643005 4336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:29.643941 4336 raft_consensus.cc:504] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c644028baf4a488badd04e26b5048286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } }
I20260923 18:04:29.644248 4336 leader_election.cc:302] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c644028baf4a488badd04e26b5048286; no voters:
I20260923 18:04:29.644496 4336 leader_election.cc:288] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:29.644645 4346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:29.644894 4346 raft_consensus.cc:686] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [term 1 LEADER]: Becoming Leader. State: Replica: c644028baf4a488badd04e26b5048286, State: Running, Role: LEADER
I20260923 18:04:29.645274 4346 consensus_queue.cc:237] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c644028baf4a488badd04e26b5048286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } }
I20260923 18:04:29.645710 4336 sys_catalog.cc:564] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:29.646999 4347 sys_catalog.cc:455] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c644028baf4a488badd04e26b5048286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c644028baf4a488badd04e26b5048286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } } }
I20260923 18:04:29.647120 4347 sys_catalog.cc:458] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:29.647542 4348 sys_catalog.cc:455] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c644028baf4a488badd04e26b5048286. Latest consensus state: current_term: 1 leader_uuid: "c644028baf4a488badd04e26b5048286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c644028baf4a488badd04e26b5048286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } } }
I20260923 18:04:29.647632 4348 sys_catalog.cc:458] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286 [sys.catalog]: This master's current role is: LEADER
W20260923 18:04:29.650420 4361 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:04:29.650537 4361 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:04:29.650663 4355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:29.651312 4355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:29.657335 4355 catalog_manager.cc:1347] Generated new cluster ID: 0224a6672b4b4e0db7091b61e387805e
I20260923 18:04:29.657415 4355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:29.680227 4355 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:29.680580 4355 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:29.689157 4355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c644028baf4a488badd04e26b5048286: Generated new TSK 0
I20260923 18:04:29.690007 4355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:29.706028 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 4268
2026-09-23T18:04:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1457 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:29 dist-test-slave-vr57 krb5kdc[4367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:29 dist-test-slave-vr57 krb5kdc[4367](info): set up 2 sockets
Sep 23 18:04:29 dist-test-slave-vr57 krb5kdc[4367](info): commencing operation
krb5kdc: starting...
W20260923 18:04:30.748528 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:30 dist-test-slave-vr57 krb5kdc[4367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 18:04:30.875159 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43633
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:30.991997 4380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:30.992286 4380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:30.992365 4380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:30.996623 4380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:30.996701 4380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:30.996721 4380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:30.996739 4380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:30.996780 4380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:31.002489 4380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:31.003721 4380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:31.004781 4380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:31.013566 4385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:31.013587 4388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:31.013566 4386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:31.013976 4380 server_base.cc:1034] running on GCE node
I20260923 18:04:31.014523 4380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:31.015697 4380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:31.016887 4380 hybrid_clock.cc:648] HybridClock initialized: now 1790186671016867 us; error 33 us; skew 500 ppm
Sep 23 18:04:31 dist-test-slave-vr57 krb5kdc[4367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186671, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:31.029168 4380 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 18:04:31.030150 4380 webserver.cc:460] Webserver started at http://127.0.0.1:38217/ using document root <none> and password file <none>
I20260923 18:04:31.030697 4380 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:31.030758 4380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:31.030925 4380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:31.033124 4380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "9442afa9cc8b4a39af54ef06bcedcd8e"
format_stamp: "Formatted at 2026-09-23 18:04:31 on dist-test-slave-vr57"
server_key: "b340ac2ed53613780033d81970b59ac0"
server_key_iv: "95b3499df63c700cbf8c0c44a517a05a"
server_key_version: "encryptionkey@0"
I20260923 18:04:31.033632 4380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "9442afa9cc8b4a39af54ef06bcedcd8e"
format_stamp: "Formatted at 2026-09-23 18:04:31 on dist-test-slave-vr57"
server_key: "b340ac2ed53613780033d81970b59ac0"
server_key_iv: "95b3499df63c700cbf8c0c44a517a05a"
server_key_version: "encryptionkey@0"
I20260923 18:04:31.037467 4380 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260923 18:04:31.040021 4395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:31.041321 4380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:04:31.041456 4380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "9442afa9cc8b4a39af54ef06bcedcd8e"
format_stamp: "Formatted at 2026-09-23 18:04:31 on dist-test-slave-vr57"
server_key: "b340ac2ed53613780033d81970b59ac0"
server_key_iv: "95b3499df63c700cbf8c0c44a517a05a"
server_key_version: "encryptionkey@0"
I20260923 18:04:31.041563 4380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:31.053562 4380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:31.056782 4380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:31.057005 4380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:31.064194 4380 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43633
I20260923 18:04:31.064230 4447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43633 every 8 connection(s)
I20260923 18:04:31.065153 4380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:31.067986 4448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:31.071956 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 4380
I20260923 18:04:31.072067 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:31.072430 15918 external_mini_cluster.cc:1428] Setting key 996a8604ff1c39522a19f2335a9fb0ea
I20260923 18:04:31.073786 4448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e: Bootstrap starting.
I20260923 18:04:31.076150 4448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:31.076916 4448 log.cc:824] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:31.078804 4448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e: No bootstrap required, opened a new log
Sep 23 18:04:31 dist-test-slave-vr57 krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 18:04:31.082479 4448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9442afa9cc8b4a39af54ef06bcedcd8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43633 } }
I20260923 18:04:31.082739 4448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:31.082830 4448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9442afa9cc8b4a39af54ef06bcedcd8e, State: Initialized, Role: FOLLOWER
I20260923 18:04:31.083314 4448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9442afa9cc8b4a39af54ef06bcedcd8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43633 } }
I20260923 18:04:31.083468 4448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:31.083593 4448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:31.083699 4448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:31.084590 4448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9442afa9cc8b4a39af54ef06bcedcd8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43633 } }
I20260923 18:04:31.084935 4448 leader_election.cc:302] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9442afa9cc8b4a39af54ef06bcedcd8e; no voters:
I20260923 18:04:31.085268 4448 leader_election.cc:288] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:31.085357 4453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:31.085564 4453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [term 1 LEADER]: Becoming Leader. State: Replica: 9442afa9cc8b4a39af54ef06bcedcd8e, State: Running, Role: LEADER
I20260923 18:04:31.085937 4453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9442afa9cc8b4a39af54ef06bcedcd8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43633 } }
I20260923 18:04:31.086623 4448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:31.087746 4455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9442afa9cc8b4a39af54ef06bcedcd8e. Latest consensus state: current_term: 1 leader_uuid: "9442afa9cc8b4a39af54ef06bcedcd8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9442afa9cc8b4a39af54ef06bcedcd8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43633 } } }
I20260923 18:04:31.087858 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:31.073900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:53081 (local address 127.0.0.1:43633)
0923 18:04:31.074354 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:31.074369 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:31.074405 (+ 36us) server_negotiation.cc:407] Connection header received
0923 18:04:31.075066 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:31.075082 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:31.075437 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:31.075860 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:31.076707 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:31.077737 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:31.078652 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:31.079186 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:31.079409 (+ 223us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:31.082011 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:31.082015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:31.082029 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:31.082055 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:31.084096 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:31.084597 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:31.084611 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:31.084617 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:31.084766 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:31.085112 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:31.085116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:31.085118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:31.085687 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:31.085898 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:31.086540 (+ 642us) server_negotiation.cc:299] Negotiation successful
0923 18:04:31.086776 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":123,"threads_started":1}
I20260923 18:04:31.087884 4455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:31.087992 4454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9442afa9cc8b4a39af54ef06bcedcd8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9442afa9cc8b4a39af54ef06bcedcd8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43633 } } }
I20260923 18:04:31.088068 4454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:31.088323 4462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:31.091394 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:31.091472 4462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:31.097555 4462 catalog_manager.cc:1347] Generated new cluster ID: 27dd137700334de18a7b606fd8234901
I20260923 18:04:31.097723 4462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:31.105491 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:31.093870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:37503 (local address 127.0.0.1:43633)
0923 18:04:31.094015 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:31.094021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:31.094498 (+ 477us) server_negotiation.cc:407] Connection header received
0923 18:04:31.094679 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:31.094683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:31.094748 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:31.094860 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:31.095790 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:31.096630 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:31.097820 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:31.098764 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:31.098869 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:31.100931 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:31.100936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:31.100939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:31.100968 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:31.102818 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:31.103458 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:31.103462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:31.103464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:31.103574 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:31.103980 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:31.103983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:31.103985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:31.104228 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:31.104369 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:31.105214 (+ 845us) server_negotiation.cc:299] Negotiation successful
0923 18:04:31.105319 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:04:31.112738 4462 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:31.113724 4462 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:31.119455 4462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9442afa9cc8b4a39af54ef06bcedcd8e: Generated new TSK 0
I20260923 18:04:31.120340 4462 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:31.127373 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 4380
W20260923 18:04:31.133095 4365 connection.cc:535] client connection to 127.0.0.1:43633 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43633): Connection reset by peer (error 104)
2026-09-23T18:04:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:31 dist-test-slave-vr57 krb5kdc[4479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:31 dist-test-slave-vr57 krb5kdc[4479](info): set up 2 sockets
Sep 23 18:04:31 dist-test-slave-vr57 krb5kdc[4479](info): commencing operation
krb5kdc: starting...
W20260923 18:04:32.180338 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:32 dist-test-slave-vr57 krb5kdc[4479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 18:04:32.312073 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41271
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:32.426604 4492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:32.427268 4492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:32.427412 4492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:32.432381 4492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:32.432492 4492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:32.432547 4492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:32.432588 4492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:32.432628 4492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:32.438305 4492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:32.439558 4492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:32.440659 4492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:32.449514 4500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:32.449529 4497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:32.449512 4498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:32.449915 4492 server_base.cc:1034] running on GCE node
I20260923 18:04:32.450438 4492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:32.451586 4492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:32.452773 4492 hybrid_clock.cc:648] HybridClock initialized: now 1790186672452754 us; error 33 us; skew 500 ppm
Sep 23 18:04:32 dist-test-slave-vr57 krb5kdc[4479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186672, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:32.455547 4492 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 18:04:32.456441 4492 webserver.cc:460] Webserver started at http://127.0.0.1:34643/ using document root <none> and password file <none>
I20260923 18:04:32.456965 4492 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:32.457047 4492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:32.457219 4492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:32.459249 4492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "fa97e9913e044b1faacb2df5b35b64c7"
format_stamp: "Formatted at 2026-09-23 18:04:32 on dist-test-slave-vr57"
server_key: "c9f25cb38ad3abb21c5538b95b838108"
server_key_iv: "9a4211dba22499d2d822aef5753b2658"
server_key_version: "encryptionkey@0"
I20260923 18:04:32.459895 4492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "fa97e9913e044b1faacb2df5b35b64c7"
format_stamp: "Formatted at 2026-09-23 18:04:32 on dist-test-slave-vr57"
server_key: "c9f25cb38ad3abb21c5538b95b838108"
server_key_iv: "9a4211dba22499d2d822aef5753b2658"
server_key_version: "encryptionkey@0"
I20260923 18:04:32.463454 4492 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:32.465956 4507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:32.467233 4492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:04:32.467334 4492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "fa97e9913e044b1faacb2df5b35b64c7"
format_stamp: "Formatted at 2026-09-23 18:04:32 on dist-test-slave-vr57"
server_key: "c9f25cb38ad3abb21c5538b95b838108"
server_key_iv: "9a4211dba22499d2d822aef5753b2658"
server_key_version: "encryptionkey@0"
I20260923 18:04:32.467419 4492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:32.493726 4492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:32.496877 4492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:32.497035 4492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:32.504020 4492 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41271
I20260923 18:04:32.504046 4559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41271 every 8 connection(s)
I20260923 18:04:32.504958 4492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:32.507614 4560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:32.509446 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 4492
I20260923 18:04:32.509624 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:32.509924 15918 external_mini_cluster.cc:1428] Setting key e3d87699a0f98198367f129371a9ab22
I20260923 18:04:32.514055 4560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7: Bootstrap starting.
I20260923 18:04:32.516314 4560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:32.517162 4560 log.cc:824] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:32.519009 4563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:32.511322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:47901 (local address 127.0.0.1:41271)
0923 18:04:32.511798 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:32.511812 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:32.511847 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:04:32.512480 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:32.512497 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:32.512834 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:32.513188 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:32.513590 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:32.514640 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:32.515559 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:32.516038 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:32.516211 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:32.517119 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:32.517126 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:32.517140 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:32.517601 (+ 461us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:32.517678 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:32.517712 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:32.517852 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:32.518103 (+ 251us) server_negotiation.cc:299] Negotiation successful
0923 18:04:32.518260 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":152,"threads_started":1}
I20260923 18:04:32.519178 4560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7: No bootstrap required, opened a new log
I20260923 18:04:32.522001 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:32.523605 4560 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa97e9913e044b1faacb2df5b35b64c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } }
I20260923 18:04:32.523883 4560 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:32.523952 4560 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa97e9913e044b1faacb2df5b35b64c7, State: Initialized, Role: FOLLOWER
I20260923 18:04:32.524456 4560 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa97e9913e044b1faacb2df5b35b64c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } }
I20260923 18:04:32.524622 4560 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:32.524708 4560 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:32.524811 4560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:32.525956 4560 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa97e9913e044b1faacb2df5b35b64c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } }
I20260923 18:04:32.526321 4560 leader_election.cc:302] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa97e9913e044b1faacb2df5b35b64c7; no voters:
I20260923 18:04:32.526654 4560 leader_election.cc:288] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:32.526746 4570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:32.527004 4570 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [term 1 LEADER]: Becoming Leader. State: Replica: fa97e9913e044b1faacb2df5b35b64c7, State: Running, Role: LEADER
I20260923 18:04:32.527359 4570 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa97e9913e044b1faacb2df5b35b64c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } }
I20260923 18:04:32.528040 4560 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:32.529037 4572 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa97e9913e044b1faacb2df5b35b64c7. Latest consensus state: current_term: 1 leader_uuid: "fa97e9913e044b1faacb2df5b35b64c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa97e9913e044b1faacb2df5b35b64c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } } }
I20260923 18:04:32.529173 4572 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:32.529556 4576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:32.529528 4571 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa97e9913e044b1faacb2df5b35b64c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa97e9913e044b1faacb2df5b35b64c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } } }
I20260923 18:04:32.529613 4571 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:32.531179 4563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:32.524311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:46049 (local address 127.0.0.1:41271)
0923 18:04:32.524435 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:32.524452 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:32.524666 (+ 214us) server_negotiation.cc:407] Connection header received
0923 18:04:32.524834 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:32.524837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:32.524893 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:32.524994 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:32.525306 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:32.526076 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:32.527297 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:32.527640 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:32.527715 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:32.530667 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:32.530671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:32.530674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:32.530690 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:32.530722 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:32.530741 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:32.530841 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:32.531012 (+ 171us) server_negotiation.cc:299] Negotiation successful
0923 18:04:32.531047 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 18:04:32.532732 4576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:32.538467 4576 catalog_manager.cc:1347] Generated new cluster ID: 748a8b318af547eab1c80ad15b1649f9
I20260923 18:04:32.538546 4576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:32.553975 4576 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:32.555251 4576 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:32.562844 4576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa97e9913e044b1faacb2df5b35b64c7: Generated new TSK 0
I20260923 18:04:32.563706 4576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:32.572994 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 4492
W20260923 18:04:32.578612 4477 connection.cc:535] client connection to 127.0.0.1:41271 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41271): Connection reset by peer (error 104)
2026-09-23T18:04:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1445 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:32 dist-test-slave-vr57 krb5kdc[4591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:32 dist-test-slave-vr57 krb5kdc[4591](info): set up 2 sockets
Sep 23 18:04:32 dist-test-slave-vr57 krb5kdc[4591](info): commencing operation
krb5kdc: starting...
W20260923 18:04:33.648259 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:33 dist-test-slave-vr57 krb5kdc[4591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 18:04:33.770223 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32829
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:33.881313 4604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:33.881582 4604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:33.881632 4604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:33.885691 4604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:33.885757 4604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:33.885777 4604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:33.885795 4604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:33.885810 4604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:33.890990 4604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:33.892089 4604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:33.893080 4604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:33.901882 4610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:33.901916 4609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:33.902047 4604 server_base.cc:1034] running on GCE node
W20260923 18:04:33.902232 4612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:33.902760 4604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:33.903910 4604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:33.905128 4604 hybrid_clock.cc:648] HybridClock initialized: now 1790186673905090 us; error 32 us; skew 500 ppm
Sep 23 18:04:33 dist-test-slave-vr57 krb5kdc[4591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186673, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:33.907828 4604 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 18:04:33.908651 4604 webserver.cc:460] Webserver started at http://127.0.0.1:36105/ using document root <none> and password file <none>
I20260923 18:04:33.909193 4604 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:33.909252 4604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:33.909415 4604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:33.911427 4604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "62110563254a4f05a48e5c6f539cc3b6"
format_stamp: "Formatted at 2026-09-23 18:04:33 on dist-test-slave-vr57"
server_key: "8367c1566e879d34664c3f48d4702c3e"
server_key_iv: "784525e2e9b121c9d6f2c80958859646"
server_key_version: "encryptionkey@0"
I20260923 18:04:33.912029 4604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "62110563254a4f05a48e5c6f539cc3b6"
format_stamp: "Formatted at 2026-09-23 18:04:33 on dist-test-slave-vr57"
server_key: "8367c1566e879d34664c3f48d4702c3e"
server_key_iv: "784525e2e9b121c9d6f2c80958859646"
server_key_version: "encryptionkey@0"
I20260923 18:04:33.915656 4604 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:33.918025 4619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:33.919246 4604 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:04:33.919344 4604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "62110563254a4f05a48e5c6f539cc3b6"
format_stamp: "Formatted at 2026-09-23 18:04:33 on dist-test-slave-vr57"
server_key: "8367c1566e879d34664c3f48d4702c3e"
server_key_iv: "784525e2e9b121c9d6f2c80958859646"
server_key_version: "encryptionkey@0"
I20260923 18:04:33.919430 4604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:33.923955 4604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:33.927124 4604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:33.927321 4604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:33.934187 4604 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32829
I20260923 18:04:33.934259 4671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32829 every 8 connection(s)
I20260923 18:04:33.935134 4604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:33.936544 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 4604
I20260923 18:04:33.936671 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:33.936997 15918 external_mini_cluster.cc:1428] Setting key a94deb7c44adb71e4c661562fe5a0614
I20260923 18:04:33.938359 4672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:33.942214 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:33.938421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:41845 (local address 127.0.0.1:32829)
0923 18:04:33.938859 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:33.938874 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:33.938909 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:04:33.939628 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:33.939645 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:33.939922 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:33.940270 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:33.941040 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:33.941044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:33.941057 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:33.941067 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:33.941120 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:33.941139 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:33.941298 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:33.941368 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 18:04:33.941531 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":148,"threads_started":1}
I20260923 18:04:33.944710 4672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6: Bootstrap starting.
I20260923 18:04:33.945097 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:33.947113 4672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:33.947984 4672 log.cc:824] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:33.948874 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:33.947115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:40357 (local address 127.0.0.1:32829)
0923 18:04:33.947265 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:33.947271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:33.947471 (+ 200us) server_negotiation.cc:407] Connection header received
0923 18:04:33.947625 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:33.947628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:33.947667 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:33.947754 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:33.948463 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:33.948467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:33.948469 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:33.948480 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:33.948507 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:33.948526 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:33.948612 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:33.948717 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 18:04:33.948758 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 18:04:33.950075 4672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6: No bootstrap required, opened a new log
I20260923 18:04:33.953454 4672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62110563254a4f05a48e5c6f539cc3b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32829 } }
I20260923 18:04:33.953677 4672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:33.953719 4672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62110563254a4f05a48e5c6f539cc3b6, State: Initialized, Role: FOLLOWER
I20260923 18:04:33.954123 4672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62110563254a4f05a48e5c6f539cc3b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32829 } }
I20260923 18:04:33.954293 4672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:33.954428 4672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:33.954545 4672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:33.955425 4672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62110563254a4f05a48e5c6f539cc3b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32829 } }
I20260923 18:04:33.955789 4672 leader_election.cc:302] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62110563254a4f05a48e5c6f539cc3b6; no voters:
I20260923 18:04:33.956089 4672 leader_election.cc:288] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:33.956218 4682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:33.956446 4682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [term 1 LEADER]: Becoming Leader. State: Replica: 62110563254a4f05a48e5c6f539cc3b6, State: Running, Role: LEADER
I20260923 18:04:33.956792 4682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62110563254a4f05a48e5c6f539cc3b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32829 } }
I20260923 18:04:33.957360 4672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:33.958690 4684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62110563254a4f05a48e5c6f539cc3b6. Latest consensus state: current_term: 1 leader_uuid: "62110563254a4f05a48e5c6f539cc3b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62110563254a4f05a48e5c6f539cc3b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32829 } } }
I20260923 18:04:33.958734 4683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62110563254a4f05a48e5c6f539cc3b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62110563254a4f05a48e5c6f539cc3b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32829 } } }
I20260923 18:04:33.958834 4683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:33.958834 4684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:33.959751 4692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:33.962745 4692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:33.968060 4692 catalog_manager.cc:1347] Generated new cluster ID: 01771fa46f2f49c5a169b8a8440cd7b9
I20260923 18:04:33.968133 4692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:33.976626 4692 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:33.976967 4692 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:33.981988 4692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62110563254a4f05a48e5c6f539cc3b6: Generated new TSK 0
I20260923 18:04:33.982795 4692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:33.992605 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 4604
2026-09-23T18:04:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1419 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:34 dist-test-slave-vr57 krb5kdc[4703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:34 dist-test-slave-vr57 krb5kdc[4703](info): set up 2 sockets
Sep 23 18:04:34 dist-test-slave-vr57 krb5kdc[4703](info): commencing operation
krb5kdc: starting...
W20260923 18:04:35.032374 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:35 dist-test-slave-vr57 krb5kdc[4703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 18:04:35.159722 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:35.271066 4716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:35.271401 4716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:35.271485 4716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:35.275590 4716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:35.275695 4716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:35.275733 4716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:35.275794 4716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:35.275831 4716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:35.281131 4716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42853
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:35.282262 4716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:35.283313 4716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:35.292632 4722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:35.292677 4724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:35.292594 4721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:35.292845 4716 server_base.cc:1034] running on GCE node
I20260923 18:04:35.293529 4716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:35.294691 4716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:35.295857 4716 hybrid_clock.cc:648] HybridClock initialized: now 1790186675295820 us; error 55 us; skew 500 ppm
Sep 23 18:04:35 dist-test-slave-vr57 krb5kdc[4703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186675, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:35.298779 4716 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 18:04:35.299777 4716 webserver.cc:460] Webserver started at http://127.0.0.1:43799/ using document root <none> and password file <none>
I20260923 18:04:35.300343 4716 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:35.300405 4716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:35.300647 4716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:35.302702 4716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "48ecb3d459214718a24a0b389575d122"
format_stamp: "Formatted at 2026-09-23 18:04:35 on dist-test-slave-vr57"
server_key: "bb36466ad5dfccec678954b537ad3a95"
server_key_iv: "76fc06088002cd5f7e633891bb23a4ad"
server_key_version: "encryptionkey@0"
I20260923 18:04:35.303257 4716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "48ecb3d459214718a24a0b389575d122"
format_stamp: "Formatted at 2026-09-23 18:04:35 on dist-test-slave-vr57"
server_key: "bb36466ad5dfccec678954b537ad3a95"
server_key_iv: "76fc06088002cd5f7e633891bb23a4ad"
server_key_version: "encryptionkey@0"
I20260923 18:04:35.307052 4716 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:04:35.309715 4731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:35.311036 4716 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260923 18:04:35.311185 4716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "48ecb3d459214718a24a0b389575d122"
format_stamp: "Formatted at 2026-09-23 18:04:35 on dist-test-slave-vr57"
server_key: "bb36466ad5dfccec678954b537ad3a95"
server_key_iv: "76fc06088002cd5f7e633891bb23a4ad"
server_key_version: "encryptionkey@0"
I20260923 18:04:35.311302 4716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:35.326858 4716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:35.330384 4716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:35.330602 4716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:35.338109 4716 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42853
I20260923 18:04:35.338145 4783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42853 every 8 connection(s)
I20260923 18:04:35.339058 4716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:35.342022 4784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:35.346395 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 4716
I20260923 18:04:35.346509 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:35.346800 15918 external_mini_cluster.cc:1428] Setting key 911c6c40fff5e6c64da37e9f1d8710bf
I20260923 18:04:35.348587 4784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122: Bootstrap starting.
I20260923 18:04:35.351568 4784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:35.352536 4784 log.cc:824] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:35.354730 4784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122: No bootstrap required, opened a new log
Sep 23 18:04:35 dist-test-slave-vr57 krb5kdc[4703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 18:04:35.359547 4784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ecb3d459214718a24a0b389575d122" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42853 } }
I20260923 18:04:35.359889 4784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:35.359959 4784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48ecb3d459214718a24a0b389575d122, State: Initialized, Role: FOLLOWER
I20260923 18:04:35.360589 4784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ecb3d459214718a24a0b389575d122" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42853 } }
I20260923 18:04:35.360780 4784 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:35.360860 4784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:35.360975 4784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:35.362313 4784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ecb3d459214718a24a0b389575d122" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42853 } }
I20260923 18:04:35.362726 4784 leader_election.cc:302] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48ecb3d459214718a24a0b389575d122; no voters:
I20260923 18:04:35.363111 4784 leader_election.cc:288] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:35.363272 4789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:35.363507 4789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [term 1 LEADER]: Becoming Leader. State: Replica: 48ecb3d459214718a24a0b389575d122, State: Running, Role: LEADER
I20260923 18:04:35.363597 4787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:35.348321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:49517 (local address 127.0.0.1:42853)
0923 18:04:35.348785 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:35.348811 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:35.348861 (+ 50us) server_negotiation.cc:407] Connection header received
0923 18:04:35.349762 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:35.349790 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:35.350288 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:35.350678 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:35.351350 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:35.353081 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:35.353992 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:35.354730 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:35.354908 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:35.357550 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:35.357560 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:35.357586 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:35.357634 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:35.359879 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:35.360455 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:35.360461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:35.360467 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:35.360642 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:35.361045 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:35.361050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:35.361052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:35.361656 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:35.361896 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:35.362135 (+ 239us) server_negotiation.cc:299] Negotiation successful
0923 18:04:35.362403 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":304,"thread_start_us":126,"threads_started":1}
I20260923 18:04:35.363938 4789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ecb3d459214718a24a0b389575d122" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42853 } }
I20260923 18:04:35.364588 4784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:35.365176 4789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48ecb3d459214718a24a0b389575d122. Latest consensus state: current_term: 1 leader_uuid: "48ecb3d459214718a24a0b389575d122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ecb3d459214718a24a0b389575d122" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42853 } } }
I20260923 18:04:35.365302 4789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:35.365604 4791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48ecb3d459214718a24a0b389575d122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ecb3d459214718a24a0b389575d122" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42853 } } }
I20260923 18:04:35.365708 4791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:35.365775 4793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:35.367631 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:35.369760 4793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:35.375631 4793 catalog_manager.cc:1347] Generated new cluster ID: 50b9cd02b3db4c70857d38c3be05d013
I20260923 18:04:35.375744 4793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:35.385294 4787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:35.372009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:33973 (local address 127.0.0.1:42853)
0923 18:04:35.372122 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:35.372127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:35.375849 (+ 3722us) server_negotiation.cc:407] Connection header received
0923 18:04:35.376095 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:35.376099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:35.376156 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:35.376252 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:35.377202 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:35.378027 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:35.379259 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:35.379674 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:35.379776 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:35.381557 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:35.381561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:35.381564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:35.381592 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:35.383321 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:35.383909 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:35.383913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:35.383915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:35.384001 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:35.384347 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:35.384351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:35.384353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:35.384564 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:35.384708 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:35.385024 (+ 316us) server_negotiation.cc:299] Negotiation successful
0923 18:04:35.385125 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 18:04:35.397805 4793 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:35.398849 4793 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:35.408416 4793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48ecb3d459214718a24a0b389575d122: Generated new TSK 0
I20260923 18:04:35.409219 4793 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:35.414242 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 4716
W20260923 18:04:35.419396 4701 connection.cc:535] client connection to 127.0.0.1:42853 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42853): Connection reset by peer (error 104)
2026-09-23T18:04:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:35 dist-test-slave-vr57 krb5kdc[4815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:35 dist-test-slave-vr57 krb5kdc[4815](info): set up 2 sockets
Sep 23 18:04:35 dist-test-slave-vr57 krb5kdc[4815](info): commencing operation
krb5kdc: starting...
W20260923 18:04:36.508422 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:36 dist-test-slave-vr57 krb5kdc[4815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 18:04:36.635222 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34373
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:36.750789 4828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:36.751087 4828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:36.751133 4828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:36.755214 4828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:36.755291 4828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:36.755316 4828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:36.755378 4828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:36.755417 4828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:36.760800 4828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34373
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:36.761932 4828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:36.763007 4828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:36.771795 4833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:36.771795 4836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:36.771790 4834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:36.772132 4828 server_base.cc:1034] running on GCE node
I20260923 18:04:36.772801 4828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:36.773998 4828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:36.775197 4828 hybrid_clock.cc:648] HybridClock initialized: now 1790186676775167 us; error 45 us; skew 500 ppm
Sep 23 18:04:36 dist-test-slave-vr57 krb5kdc[4815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:36.786579 4828 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 18:04:36.787690 4828 webserver.cc:460] Webserver started at http://127.0.0.1:42665/ using document root <none> and password file <none>
I20260923 18:04:36.788301 4828 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:36.788398 4828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:36.788645 4828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:36.790740 4828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "79b7bfa123d44ac580dcd2e94d7ef007"
format_stamp: "Formatted at 2026-09-23 18:04:36 on dist-test-slave-vr57"
server_key: "5046c7623a4ca2462f853d88e897ce5c"
server_key_iv: "74e9b4d01fe71f291eece1f7993bafd7"
server_key_version: "encryptionkey@0"
I20260923 18:04:36.791330 4828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "79b7bfa123d44ac580dcd2e94d7ef007"
format_stamp: "Formatted at 2026-09-23 18:04:36 on dist-test-slave-vr57"
server_key: "5046c7623a4ca2462f853d88e897ce5c"
server_key_iv: "74e9b4d01fe71f291eece1f7993bafd7"
server_key_version: "encryptionkey@0"
I20260923 18:04:36.795492 4828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260923 18:04:36.798081 4843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:36.799427 4828 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:04:36.799608 4828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "79b7bfa123d44ac580dcd2e94d7ef007"
format_stamp: "Formatted at 2026-09-23 18:04:36 on dist-test-slave-vr57"
server_key: "5046c7623a4ca2462f853d88e897ce5c"
server_key_iv: "74e9b4d01fe71f291eece1f7993bafd7"
server_key_version: "encryptionkey@0"
I20260923 18:04:36.799726 4828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:36.820075 4828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:36.823210 4828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:36.823428 4828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:36.830777 4828 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34373
I20260923 18:04:36.830804 4895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34373 every 8 connection(s)
I20260923 18:04:36.831795 4828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:36.832113 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 4828
I20260923 18:04:36.832269 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:36.832589 15918 external_mini_cluster.cc:1428] Setting key 7a6ced481066886c05af17a2c2bde476
I20260923 18:04:36.834846 4896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:36.841413 4896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007: Bootstrap starting.
I20260923 18:04:36.841996 4899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:36.834001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:51649 (local address 127.0.0.1:34373)
0923 18:04:36.834411 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:36.834426 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:36.834460 (+ 34us) server_negotiation.cc:407] Connection header received
0923 18:04:36.835166 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:36.835183 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:36.835581 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:36.836024 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:36.836435 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:36.837439 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:36.838648 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:36.839097 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:36.839220 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:36.840183 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:36.840188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:36.840205 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:36.840581 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:36.840664 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:36.840689 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:36.840790 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:36.841007 (+ 217us) server_negotiation.cc:299] Negotiation successful
0923 18:04:36.841169 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":140,"threads_started":1}
I20260923 18:04:36.843734 4896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:36.844584 4896 log.cc:824] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:36.845037 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:36.846799 4896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007: No bootstrap required, opened a new log
I20260923 18:04:36.851348 4899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:36.847186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:33111 (local address 127.0.0.1:34373)
0923 18:04:36.847341 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:36.847345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:36.847613 (+ 268us) server_negotiation.cc:407] Connection header received
0923 18:04:36.847722 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:36.847725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:36.847769 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:36.847862 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:36.848063 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:36.848819 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:36.849648 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:36.849979 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:36.850088 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:36.850918 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:36.850925 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:36.850928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:36.850941 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:36.850971 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:36.850986 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:36.851082 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:36.851184 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 18:04:36.851219 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260923 18:04:36.851730 4896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b7bfa123d44ac580dcd2e94d7ef007" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34373 } }
I20260923 18:04:36.851986 4896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:36.852067 4896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b7bfa123d44ac580dcd2e94d7ef007, State: Initialized, Role: FOLLOWER
I20260923 18:04:36.852478 4896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b7bfa123d44ac580dcd2e94d7ef007" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34373 } }
I20260923 18:04:36.852597 4896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:36.852682 4896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:36.852782 4896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:36.853744 4896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b7bfa123d44ac580dcd2e94d7ef007" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34373 } }
I20260923 18:04:36.854099 4896 leader_election.cc:302] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79b7bfa123d44ac580dcd2e94d7ef007; no voters:
I20260923 18:04:36.854429 4896 leader_election.cc:288] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:36.854547 4906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:36.854820 4906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [term 1 LEADER]: Becoming Leader. State: Replica: 79b7bfa123d44ac580dcd2e94d7ef007, State: Running, Role: LEADER
I20260923 18:04:36.855144 4906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b7bfa123d44ac580dcd2e94d7ef007" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34373 } }
I20260923 18:04:36.855947 4896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:36.856909 4907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79b7bfa123d44ac580dcd2e94d7ef007" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b7bfa123d44ac580dcd2e94d7ef007" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34373 } } }
I20260923 18:04:36.856971 4908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79b7bfa123d44ac580dcd2e94d7ef007. Latest consensus state: current_term: 1 leader_uuid: "79b7bfa123d44ac580dcd2e94d7ef007" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b7bfa123d44ac580dcd2e94d7ef007" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34373 } } }
I20260923 18:04:36.857057 4907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:36.857064 4908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:36.857990 4915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:36.860841 4915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:36.866590 4915 catalog_manager.cc:1347] Generated new cluster ID: 979d0535947344e0804191cc0b958e9b
I20260923 18:04:36.866688 4915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:36.879607 4915 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:36.880679 4915 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:36.890127 4915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79b7bfa123d44ac580dcd2e94d7ef007: Generated new TSK 0
I20260923 18:04:36.890916 4915 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:36.897806 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 4828
W20260923 18:04:36.903414 4813 connection.cc:535] client connection to 127.0.0.1:34373 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34373): Connection reset by peer (error 104)
2026-09-23T18:04:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1483 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:36 dist-test-slave-vr57 krb5kdc[4927](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:36 dist-test-slave-vr57 krb5kdc[4927](info): set up 2 sockets
Sep 23 18:04:36 dist-test-slave-vr57 krb5kdc[4927](info): commencing operation
krb5kdc: starting...
W20260923 18:04:38.056186 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:38 dist-test-slave-vr57 krb5kdc[4927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 18:04:38.178579 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44185
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:38.293088 4940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:38.293390 4940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:38.293437 4940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:38.297551 4940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:38.297623 4940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:38.297646 4940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:38.297667 4940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:38.297736 4940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:38.303018 4940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44185
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:38.304165 4940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:38.305316 4940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:38.314222 4945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:38.314222 4946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:38.314363 4948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:38.314605 4940 server_base.cc:1034] running on GCE node
I20260923 18:04:38.315306 4940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:38.316536 4940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:38.317739 4940 hybrid_clock.cc:648] HybridClock initialized: now 1790186678317727 us; error 34 us; skew 500 ppm
Sep 23 18:04:38 dist-test-slave-vr57 krb5kdc[4927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186678, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:38.320715 4940 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 18:04:38.321623 4940 webserver.cc:460] Webserver started at http://127.0.0.1:35843/ using document root <none> and password file <none>
I20260923 18:04:38.322230 4940 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:38.322328 4940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:38.322563 4940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:38.324723 4940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "97626f7190c24ee3bb2477f5262427a4"
format_stamp: "Formatted at 2026-09-23 18:04:38 on dist-test-slave-vr57"
server_key: "66660f27e04f813b65eacd16ba01cb53"
server_key_iv: "893cbc9e699d0de886fee2c76e95d33f"
server_key_version: "encryptionkey@0"
I20260923 18:04:38.325296 4940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "97626f7190c24ee3bb2477f5262427a4"
format_stamp: "Formatted at 2026-09-23 18:04:38 on dist-test-slave-vr57"
server_key: "66660f27e04f813b65eacd16ba01cb53"
server_key_iv: "893cbc9e699d0de886fee2c76e95d33f"
server_key_version: "encryptionkey@0"
I20260923 18:04:38.329447 4940 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 18:04:38.331873 4955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:38.333132 4940 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260923 18:04:38.333272 4940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "97626f7190c24ee3bb2477f5262427a4"
format_stamp: "Formatted at 2026-09-23 18:04:38 on dist-test-slave-vr57"
server_key: "66660f27e04f813b65eacd16ba01cb53"
server_key_iv: "893cbc9e699d0de886fee2c76e95d33f"
server_key_version: "encryptionkey@0"
I20260923 18:04:38.333382 4940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:38.338482 4940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:38.341750 4940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:38.341961 4940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:38.350032 4940 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44185
I20260923 18:04:38.350065 5007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44185 every 8 connection(s)
I20260923 18:04:38.351060 4940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:38.353859 5008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:38.355011 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 4940
I20260923 18:04:38.355157 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:38.355479 15918 external_mini_cluster.cc:1428] Setting key 4c4c250dca65ab114fc0e73c902be179
I20260923 18:04:38.360051 5008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4: Bootstrap starting.
I20260923 18:04:38.360992 5011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:38.357016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:54119 (local address 127.0.0.1:44185)
0923 18:04:38.357487 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:38.357509 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:38.357549 (+ 40us) server_negotiation.cc:407] Connection header received
0923 18:04:38.358211 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:38.358228 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:38.358511 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:38.358901 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:38.359791 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:38.359795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:38.359807 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:38.359818 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:38.359875 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:38.359897 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:38.360061 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:38.360135 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 18:04:38.360298 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":132,"threads_started":1}
I20260923 18:04:38.362509 5008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:38.363387 5008 log.cc:824] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:38.364156 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:38.365410 5008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4: No bootstrap required, opened a new log
I20260923 18:04:38.368054 5011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:38.366251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:36783 (local address 127.0.0.1:44185)
0923 18:04:38.366370 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:38.366373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:38.366580 (+ 207us) server_negotiation.cc:407] Connection header received
0923 18:04:38.366701 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:38.366703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:38.366732 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:38.366804 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:38.367603 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:38.367606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:38.367608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:38.367619 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:38.367644 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:38.367654 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:38.367728 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:38.367840 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 18:04:38.367874 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 18:04:38.369957 5008 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97626f7190c24ee3bb2477f5262427a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44185 } }
I20260923 18:04:38.370203 5008 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:38.370258 5008 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97626f7190c24ee3bb2477f5262427a4, State: Initialized, Role: FOLLOWER
I20260923 18:04:38.370674 5008 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97626f7190c24ee3bb2477f5262427a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44185 } }
I20260923 18:04:38.370828 5008 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:38.370918 5008 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:38.371014 5008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:38.372009 5008 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97626f7190c24ee3bb2477f5262427a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44185 } }
I20260923 18:04:38.372393 5008 leader_election.cc:302] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97626f7190c24ee3bb2477f5262427a4; no voters:
I20260923 18:04:38.372735 5008 leader_election.cc:288] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:38.372888 5018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:38.373160 5018 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [term 1 LEADER]: Becoming Leader. State: Replica: 97626f7190c24ee3bb2477f5262427a4, State: Running, Role: LEADER
I20260923 18:04:38.373529 5018 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97626f7190c24ee3bb2477f5262427a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44185 } }
I20260923 18:04:38.374050 5008 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:38.375348 5020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97626f7190c24ee3bb2477f5262427a4. Latest consensus state: current_term: 1 leader_uuid: "97626f7190c24ee3bb2477f5262427a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97626f7190c24ee3bb2477f5262427a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44185 } } }
I20260923 18:04:38.375348 5019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97626f7190c24ee3bb2477f5262427a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97626f7190c24ee3bb2477f5262427a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44185 } } }
I20260923 18:04:38.375556 5019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:38.375556 5020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:38.376009 5029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:38.379096 5029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:38.384811 5029 catalog_manager.cc:1347] Generated new cluster ID: ec9320f38dc14f889bb1e15e9a2be565
I20260923 18:04:38.384883 5029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:38.401340 5029 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:38.401703 5029 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:38.410535 5029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97626f7190c24ee3bb2477f5262427a4: Generated new TSK 0
I20260923 18:04:38.411290 5029 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:38.415743 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 4940
2026-09-23T18:04:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1517 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:38 dist-test-slave-vr57 krb5kdc[5039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:38 dist-test-slave-vr57 krb5kdc[5039](info): set up 2 sockets
Sep 23 18:04:38 dist-test-slave-vr57 krb5kdc[5039](info): commencing operation
krb5kdc: starting...
W20260923 18:04:39.524247 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:39 dist-test-slave-vr57 krb5kdc[5039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:04:39.650786 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:37859
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:34979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:37859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:39.765450 5052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:39.765817 5052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:39.765908 5052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:39.770092 5052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:39.770201 5052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:39.770238 5052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:39.770278 5052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:39.770309 5052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:39.775775 5052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:34979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:37859
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:37859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:39.776906 5052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:39.778028 5052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:39.787397 5057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:39.787423 5058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:39.787568 5052 server_base.cc:1034] running on GCE node
W20260923 18:04:39.787813 5060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:39.788368 5052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:39.789573 5052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:39.790781 5052 hybrid_clock.cc:648] HybridClock initialized: now 1790186679790744 us; error 48 us; skew 500 ppm
Sep 23 18:04:39 dist-test-slave-vr57 krb5kdc[5039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186679, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:39.793785 5052 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:04:39.794729 5052 webserver.cc:460] Webserver started at http://127.15.139.190:45231/ using document root <none> and password file <none>
I20260923 18:04:39.795315 5052 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:39.795413 5052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:39.795806 5052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:39.797979 5052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "727fe89ed34e41198c8e4e910edcef75"
format_stamp: "Formatted at 2026-09-23 18:04:39 on dist-test-slave-vr57"
server_key: "8676772ee4328b73f81f9b904cc3ef91"
server_key_iv: "7e9776ec16892afc33263df9256a0389"
server_key_version: "encryptionkey@0"
I20260923 18:04:39.798581 5052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "727fe89ed34e41198c8e4e910edcef75"
format_stamp: "Formatted at 2026-09-23 18:04:39 on dist-test-slave-vr57"
server_key: "8676772ee4328b73f81f9b904cc3ef91"
server_key_iv: "7e9776ec16892afc33263df9256a0389"
server_key_version: "encryptionkey@0"
I20260923 18:04:39.802523 5052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:04:39.805224 5067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:39.806751 5052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:04:39.806912 5052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "727fe89ed34e41198c8e4e910edcef75"
format_stamp: "Formatted at 2026-09-23 18:04:39 on dist-test-slave-vr57"
server_key: "8676772ee4328b73f81f9b904cc3ef91"
server_key_iv: "7e9776ec16892afc33263df9256a0389"
server_key_version: "encryptionkey@0"
I20260923 18:04:39.807067 5052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:39.817015 5052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:39.820345 5052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:39.820569 5052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:39.828035 5052 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:37859
I20260923 18:04:39.828049 5119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:37859 every 8 connection(s)
I20260923 18:04:39.829015 5052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:39.832185 5120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:39.837738 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 5052
I20260923 18:04:39.837894 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:39.838145 5120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75: Bootstrap starting.
I20260923 18:04:39.838183 15918 external_mini_cluster.cc:1428] Setting key ac5c5d04ce18a159d235b1ba66e9c5bb
I20260923 18:04:39.840744 5120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:39.841497 5120 log.cc:824] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:39.843557 5120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75: No bootstrap required, opened a new log
Sep 23 18:04:39 dist-test-slave-vr57 krb5kdc[5039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:39.847743 5120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "727fe89ed34e41198c8e4e910edcef75" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37859 } }
I20260923 18:04:39.848026 5120 raft_consensus.cc:374] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:39.848105 5120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 727fe89ed34e41198c8e4e910edcef75, State: Initialized, Role: FOLLOWER
I20260923 18:04:39.848594 5120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "727fe89ed34e41198c8e4e910edcef75" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37859 } }
I20260923 18:04:39.848748 5120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:39.848837 5120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:39.848935 5120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:39.849817 5120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "727fe89ed34e41198c8e4e910edcef75" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37859 } }
I20260923 18:04:39.850250 5120 leader_election.cc:302] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 727fe89ed34e41198c8e4e910edcef75; no voters:
I20260923 18:04:39.850633 5120 leader_election.cc:288] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:39.850745 5125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:39.850967 5125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [term 1 LEADER]: Becoming Leader. State: Replica: 727fe89ed34e41198c8e4e910edcef75, State: Running, Role: LEADER
I20260923 18:04:39.851349 5125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "727fe89ed34e41198c8e4e910edcef75" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37859 } }
I20260923 18:04:39.851934 5120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:39.853246 5127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 727fe89ed34e41198c8e4e910edcef75. Latest consensus state: current_term: 1 leader_uuid: "727fe89ed34e41198c8e4e910edcef75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "727fe89ed34e41198c8e4e910edcef75" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37859 } } }
I20260923 18:04:39.853371 5127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:39.853605 5126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "727fe89ed34e41198c8e4e910edcef75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "727fe89ed34e41198c8e4e910edcef75" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37859 } } }
I20260923 18:04:39.853711 5126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:39.853781 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:39.839547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48309 (local address 127.15.139.190:37859)
0923 18:04:39.839976 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:39.839991 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:39.840026 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:04:39.840679 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:39.840695 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:39.841084 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:39.841423 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:39.842266 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:39.843380 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:39.844343 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:39.844822 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:39.844991 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:39.847964 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:39.847968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:39.847981 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:39.848034 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:39.850008 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:39.850700 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:39.850706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:39.850712 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:39.850865 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:39.851328 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:39.851331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:39.851333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:39.851821 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:39.852052 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:39.852415 (+ 363us) server_negotiation.cc:299] Negotiation successful
0923 18:04:39.852843 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":128,"threads_started":1}
W20260923 18:04:39.857113 5140 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:04:39.857223 5140 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:04:39.857326 5135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:39.857386 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:39.858345 5135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:39.864878 5135 catalog_manager.cc:1347] Generated new cluster ID: 004a376404f2421d87d1334636ffb5b9
I20260923 18:04:39.864948 5135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:39.867430 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:39.859329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52467 (local address 127.15.139.190:37859)
0923 18:04:39.859584 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:39.859590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:39.859644 (+ 54us) server_negotiation.cc:407] Connection header received
0923 18:04:39.859800 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:39.859803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:39.859844 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:39.859936 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:39.860596 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:39.861119 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:39.861915 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:39.862165 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:39.862271 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:39.863988 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:39.863992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:39.863994 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:39.864019 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:39.865624 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:39.866095 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:39.866099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:39.866101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:39.866185 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:39.866513 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:39.866516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:39.866518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:39.866689 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:39.866842 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:39.867110 (+ 268us) server_negotiation.cc:299] Negotiation successful
0923 18:04:39.867258 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
I20260923 18:04:39.879356 5135 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:39.880649 5135 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:39.896624 5135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 727fe89ed34e41198c8e4e910edcef75: Generated new TSK 0
I20260923 18:04:39.897432 5135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:39.904935 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 5052
W20260923 18:04:39.910454 5037 connection.cc:535] client connection to 127.15.139.190:37859 recv error: Network error: failed to read from TLS socket (remote: 127.15.139.190:37859): Connection reset by peer (error 104)
2026-09-23T18:04:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:39 dist-test-slave-vr57 krb5kdc[5151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:39 dist-test-slave-vr57 krb5kdc[5151](info): set up 2 sockets
Sep 23 18:04:39 dist-test-slave-vr57 krb5kdc[5151](info): commencing operation
krb5kdc: starting...
W20260923 18:04:41.004245 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:41 dist-test-slave-vr57 krb5kdc[5151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:04:41.130722 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:37591
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:37591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:41.244212 5164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:41.244536 5164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:41.244642 5164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:41.248911 5164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:41.249004 5164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:41.249042 5164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:41.249097 5164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:41.249137 5164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:41.254626 5164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:37591
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:37591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:41.255926 5164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:41.256978 5164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 18:04:41.265753 5164 server_base.cc:1034] running on GCE node
W20260923 18:04:41.265717 5170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:41.265745 5169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:41.265692 5172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:41.266654 5164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:41.267804 5164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:41.268991 5164 hybrid_clock.cc:648] HybridClock initialized: now 1790186681268978 us; error 25 us; skew 500 ppm
Sep 23 18:04:41 dist-test-slave-vr57 krb5kdc[5151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186681, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:41.271785 5164 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:04:41.272655 5164 webserver.cc:460] Webserver started at http://127.15.139.190:36489/ using document root <none> and password file <none>
I20260923 18:04:41.273257 5164 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:41.273321 5164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:41.273633 5164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:41.275775 5164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "37063c7a9f1d48efb2a9cd9cea0aacd5"
format_stamp: "Formatted at 2026-09-23 18:04:41 on dist-test-slave-vr57"
server_key: "a57d774399bc4c36054aa7743b75edf1"
server_key_iv: "2ad41799fecd3941baf81484d37e2504"
server_key_version: "encryptionkey@0"
I20260923 18:04:41.276367 5164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "37063c7a9f1d48efb2a9cd9cea0aacd5"
format_stamp: "Formatted at 2026-09-23 18:04:41 on dist-test-slave-vr57"
server_key: "a57d774399bc4c36054aa7743b75edf1"
server_key_iv: "2ad41799fecd3941baf81484d37e2504"
server_key_version: "encryptionkey@0"
I20260923 18:04:41.280229 5164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:04:41.282765 5179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:41.284271 5164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:41.284423 5164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "37063c7a9f1d48efb2a9cd9cea0aacd5"
format_stamp: "Formatted at 2026-09-23 18:04:41 on dist-test-slave-vr57"
server_key: "a57d774399bc4c36054aa7743b75edf1"
server_key_iv: "2ad41799fecd3941baf81484d37e2504"
server_key_version: "encryptionkey@0"
I20260923 18:04:41.284540 5164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:41.297559 5164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:41.300945 5164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:41.301220 5164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:41.308655 5164 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:37591
I20260923 18:04:41.308746 5231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:37591 every 8 connection(s)
I20260923 18:04:41.309752 5164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:41.312633 5232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:41.317683 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 5164
I20260923 18:04:41.317835 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:41.318115 15918 external_mini_cluster.cc:1428] Setting key 8f575d69b396661c2f608d5e115fc7db
I20260923 18:04:41.318483 5232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5: Bootstrap starting.
I20260923 18:04:41.321267 5232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:41.322154 5232 log.cc:824] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:41.324393 5232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5: No bootstrap required, opened a new log
I20260923 18:04:41.328985 5235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:41.319657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47247 (local address 127.15.139.190:37591)
0923 18:04:41.320103 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:41.320123 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:41.320168 (+ 45us) server_negotiation.cc:407] Connection header received
0923 18:04:41.320997 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:41.321021 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:41.321480 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:41.321868 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:41.322302 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:41.323694 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:41.324678 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:41.325395 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:41.325574 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:41.326560 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:41.326567 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:41.326585 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:41.327112 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:41.327186 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:41.327232 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:41.327391 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:41.327787 (+ 396us) server_negotiation.cc:299] Negotiation successful
0923 18:04:41.327967 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":136,"threads_started":1}
I20260923 18:04:41.329244 5232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37063c7a9f1d48efb2a9cd9cea0aacd5" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37591 } }
I20260923 18:04:41.329519 5232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:41.329569 5232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37063c7a9f1d48efb2a9cd9cea0aacd5, State: Initialized, Role: FOLLOWER
I20260923 18:04:41.330070 5232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37063c7a9f1d48efb2a9cd9cea0aacd5" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37591 } }
I20260923 18:04:41.330257 5232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:41.330327 5232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:41.330426 5232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:41.331631 5232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37063c7a9f1d48efb2a9cd9cea0aacd5" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37591 } }
I20260923 18:04:41.332010 5232 leader_election.cc:302] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37063c7a9f1d48efb2a9cd9cea0aacd5; no voters:
I20260923 18:04:41.332253 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:41.332396 5232 leader_election.cc:288] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:41.332469 5237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:41.332684 5237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [term 1 LEADER]: Becoming Leader. State: Replica: 37063c7a9f1d48efb2a9cd9cea0aacd5, State: Running, Role: LEADER
I20260923 18:04:41.333057 5237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37063c7a9f1d48efb2a9cd9cea0aacd5" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37591 } }
I20260923 18:04:41.333771 5232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:41.334355 5237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37063c7a9f1d48efb2a9cd9cea0aacd5. Latest consensus state: current_term: 1 leader_uuid: "37063c7a9f1d48efb2a9cd9cea0aacd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37063c7a9f1d48efb2a9cd9cea0aacd5" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37591 } } }
I20260923 18:04:41.334475 5237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:41.334834 5238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37063c7a9f1d48efb2a9cd9cea0aacd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37063c7a9f1d48efb2a9cd9cea0aacd5" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 37591 } } }
I20260923 18:04:41.334944 5238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:41.335348 5245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:41.338681 5245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:41.340114 5235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:41.334603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49383 (local address 127.15.139.190:37591)
0923 18:04:41.334749 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:41.334755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:41.335272 (+ 517us) server_negotiation.cc:407] Connection header received
0923 18:04:41.335466 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:41.335471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:41.335557 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:41.335620 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:41.336141 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:41.336887 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:41.338018 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:41.338339 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:41.338415 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:41.339567 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:41.339571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:41.339573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:41.339588 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:41.339615 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:41.339627 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:41.339696 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:41.339947 (+ 251us) server_negotiation.cc:299] Negotiation successful
0923 18:04:41.339986 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 18:04:41.347878 5245 catalog_manager.cc:1347] Generated new cluster ID: c45c002971a84e499f259e94da99abf4
I20260923 18:04:41.347965 5245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:41.355650 5245 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:41.356614 5245 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:41.365298 5245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37063c7a9f1d48efb2a9cd9cea0aacd5: Generated new TSK 0
I20260923 18:04:41.366169 5245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:41.371174 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 5164
W20260923 18:04:41.376240 5149 connection.cc:535] client connection to 127.15.139.190:37591 recv error: Network error: failed to read from TLS socket (remote: 127.15.139.190:37591): Connection reset by peer (error 104)
2026-09-23T18:04:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1465 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:41 dist-test-slave-vr57 krb5kdc[5263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:41 dist-test-slave-vr57 krb5kdc[5263](info): set up 2 sockets
Sep 23 18:04:41 dist-test-slave-vr57 krb5kdc[5263](info): commencing operation
krb5kdc: starting...
W20260923 18:04:42.404312 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:42 dist-test-slave-vr57 krb5kdc[5263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:04:42.531252 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:41857
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:36359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:41857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:42.650956 5276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:42.651288 5276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:42.651376 5276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:42.656034 5276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:42.656111 5276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:42.656131 5276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:42.656148 5276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:42.656196 5276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:42.661931 5276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:36359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:41857
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:41857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:42.663056 5276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:42.664166 5276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:42.672979 5282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:42.673023 5276 server_base.cc:1034] running on GCE node
W20260923 18:04:42.672984 5284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:42.673023 5281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:42.673748 5276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:42.674875 5276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:42.676059 5276 hybrid_clock.cc:648] HybridClock initialized: now 1790186682676027 us; error 42 us; skew 500 ppm
Sep 23 18:04:42 dist-test-slave-vr57 krb5kdc[5263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186682, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:42.685866 5276 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:04:42.686939 5276 webserver.cc:460] Webserver started at http://127.15.139.190:41687/ using document root <none> and password file <none>
I20260923 18:04:42.687634 5276 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:42.687705 5276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:42.687932 5276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:42.690069 5276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "d5927eb2ce794b8bbdba550436436bad"
format_stamp: "Formatted at 2026-09-23 18:04:42 on dist-test-slave-vr57"
server_key: "2a5d96e7c91ba521cfda0b625f6b9d6c"
server_key_iv: "14a9431d080b2e4ec913c7bf337cd070"
server_key_version: "encryptionkey@0"
I20260923 18:04:42.690675 5276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "d5927eb2ce794b8bbdba550436436bad"
format_stamp: "Formatted at 2026-09-23 18:04:42 on dist-test-slave-vr57"
server_key: "2a5d96e7c91ba521cfda0b625f6b9d6c"
server_key_iv: "14a9431d080b2e4ec913c7bf337cd070"
server_key_version: "encryptionkey@0"
I20260923 18:04:42.694617 5276 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:04:42.697204 5291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:42.698704 5276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 18:04:42.698848 5276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "d5927eb2ce794b8bbdba550436436bad"
format_stamp: "Formatted at 2026-09-23 18:04:42 on dist-test-slave-vr57"
server_key: "2a5d96e7c91ba521cfda0b625f6b9d6c"
server_key_iv: "14a9431d080b2e4ec913c7bf337cd070"
server_key_version: "encryptionkey@0"
I20260923 18:04:42.698966 5276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:42.703668 5276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:42.706914 5276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:42.707113 5276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:42.714748 5276 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:41857
I20260923 18:04:42.714803 5343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:41857 every 8 connection(s)
I20260923 18:04:42.715751 5276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:42.718216 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 5276
I20260923 18:04:42.718364 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:42.718647 15918 external_mini_cluster.cc:1428] Setting key 0077bccde3318f0be5f021487541b746
I20260923 18:04:42.718678 5344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:42.724480 5347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:42.720030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41093 (local address 127.15.139.190:41857)
0923 18:04:42.720448 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:42.720467 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:42.720510 (+ 43us) server_negotiation.cc:407] Connection header received
0923 18:04:42.721486 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:42.721510 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:42.721896 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:42.722288 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:42.723067 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:42.723072 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:42.723087 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:42.723099 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:42.723163 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:42.723190 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:42.723329 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:42.723394 (+ 65us) server_negotiation.cc:299] Negotiation successful
0923 18:04:42.723575 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":113,"threads_started":1}
I20260923 18:04:42.726224 5344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad: Bootstrap starting.
I20260923 18:04:42.727321 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:42.728700 5344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:42.729503 5344 log.cc:824] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:42.731217 5347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:42.729429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46921 (local address 127.15.139.190:41857)
0923 18:04:42.729555 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:42.729560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:42.729813 (+ 253us) server_negotiation.cc:407] Connection header received
0923 18:04:42.729955 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:42.729958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:42.729996 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:42.730087 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:42.730775 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:42.730777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:42.730779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:42.730789 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:42.730812 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:42.730822 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:42.730902 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:42.731014 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 18:04:42.731046 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 18:04:42.731925 5344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad: No bootstrap required, opened a new log
I20260923 18:04:42.735339 5344 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5927eb2ce794b8bbdba550436436bad" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41857 } }
I20260923 18:04:42.735627 5344 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:42.735692 5344 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5927eb2ce794b8bbdba550436436bad, State: Initialized, Role: FOLLOWER
I20260923 18:04:42.736096 5344 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5927eb2ce794b8bbdba550436436bad" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41857 } }
I20260923 18:04:42.736248 5344 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:42.736317 5344 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:42.736416 5344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:42.737498 5344 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5927eb2ce794b8bbdba550436436bad" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41857 } }
I20260923 18:04:42.737848 5344 leader_election.cc:302] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5927eb2ce794b8bbdba550436436bad; no voters:
I20260923 18:04:42.738175 5344 leader_election.cc:288] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:42.738319 5354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:42.738569 5354 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [term 1 LEADER]: Becoming Leader. State: Replica: d5927eb2ce794b8bbdba550436436bad, State: Running, Role: LEADER
I20260923 18:04:42.738940 5354 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5927eb2ce794b8bbdba550436436bad" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41857 } }
I20260923 18:04:42.739398 5344 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:42.740918 5355 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5927eb2ce794b8bbdba550436436bad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5927eb2ce794b8bbdba550436436bad" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41857 } } }
I20260923 18:04:42.740957 5356 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5927eb2ce794b8bbdba550436436bad. Latest consensus state: current_term: 1 leader_uuid: "d5927eb2ce794b8bbdba550436436bad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5927eb2ce794b8bbdba550436436bad" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 41857 } } }
I20260923 18:04:42.741115 5355 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:42.741124 5356 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:42.741585 5366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:42.744776 5366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:42.750609 5366 catalog_manager.cc:1347] Generated new cluster ID: f4867543116e4cd387dbd64bbbb36a44
I20260923 18:04:42.750689 5366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:42.758263 5366 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:42.758613 5366 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:42.766243 5366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5927eb2ce794b8bbdba550436436bad: Generated new TSK 0
I20260923 18:04:42.767076 5366 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:42.775243 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 5276
2026-09-23T18:04:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:42 dist-test-slave-vr57 krb5kdc[5375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:42 dist-test-slave-vr57 krb5kdc[5375](info): set up 2 sockets
Sep 23 18:04:42 dist-test-slave-vr57 krb5kdc[5375](info): commencing operation
krb5kdc: starting...
W20260923 18:04:43.832317 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:43 dist-test-slave-vr57 krb5kdc[5375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:04:43.962568 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:43067
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:34529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:43067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:44.074783 5388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:44.075088 5388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:44.075176 5388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:44.079257 5388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:44.079358 5388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:44.079398 5388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:44.079457 5388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:44.079516 5388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:44.084841 5388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:34529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:43067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:43067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:44.085958 5388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:44.087045 5388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:44.095906 5396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:44.095906 5393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:44.095918 5394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:44.096029 5388 server_base.cc:1034] running on GCE node
I20260923 18:04:44.096887 5388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:44.098142 5388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:44.099377 5388 hybrid_clock.cc:648] HybridClock initialized: now 1790186684099361 us; error 29 us; skew 500 ppm
Sep 23 18:04:44 dist-test-slave-vr57 krb5kdc[5375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186684, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:44.102451 5388 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:04:44.103479 5388 webserver.cc:460] Webserver started at http://127.15.139.190:44087/ using document root <none> and password file <none>
I20260923 18:04:44.104152 5388 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:44.104252 5388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:44.104494 5388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:44.106702 5388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "3178ca8a27bd4e4592b7bde0e12b095a"
format_stamp: "Formatted at 2026-09-23 18:04:44 on dist-test-slave-vr57"
server_key: "0401ecc7aa598f54940f1f9738a33d01"
server_key_iv: "fddeec14080528d2bd5a1bc801d59323"
server_key_version: "encryptionkey@0"
I20260923 18:04:44.107429 5388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "3178ca8a27bd4e4592b7bde0e12b095a"
format_stamp: "Formatted at 2026-09-23 18:04:44 on dist-test-slave-vr57"
server_key: "0401ecc7aa598f54940f1f9738a33d01"
server_key_iv: "fddeec14080528d2bd5a1bc801d59323"
server_key_version: "encryptionkey@0"
I20260923 18:04:44.112142 5388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260923 18:04:44.114881 5403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:44.116626 5388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 18:04:44.116806 5388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "3178ca8a27bd4e4592b7bde0e12b095a"
format_stamp: "Formatted at 2026-09-23 18:04:44 on dist-test-slave-vr57"
server_key: "0401ecc7aa598f54940f1f9738a33d01"
server_key_iv: "fddeec14080528d2bd5a1bc801d59323"
server_key_version: "encryptionkey@0"
I20260923 18:04:44.116940 5388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:44.136334 5388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:44.139940 5388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:44.140210 5388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:44.147897 5388 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:43067
I20260923 18:04:44.147915 5455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:43067 every 8 connection(s)
I20260923 18:04:44.148927 5388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:44.149276 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 5388
I20260923 18:04:44.149413 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:44.149706 15918 external_mini_cluster.cc:1428] Setting key 2e2bc6ed8073a57ebe2535bd1289172b
I20260923 18:04:44.152230 5456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 23 18:04:44 dist-test-slave-vr57 krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:44.159204 5456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a: Bootstrap starting.
I20260923 18:04:44.162094 5456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:44.163045 5456 log.cc:824] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:44.165050 5459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:44.151215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45069 (local address 127.15.139.190:43067)
0923 18:04:44.151658 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:44.151675 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:44.151710 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:04:44.152387 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:44.152415 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:44.152774 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:44.153078 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:44.154001 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:44.155031 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:44.156259 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:44.156724 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:44.156842 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:44.159846 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:44.159852 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:44.159868 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:44.159899 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:44.161889 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:44.162389 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:44.162397 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:44.162404 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:44.162545 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:44.162811 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:44.162816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:44.162820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:44.163343 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:44.163572 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:44.163826 (+ 254us) server_negotiation.cc:299] Negotiation successful
0923 18:04:44.164143 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":136,"threads_started":1}
I20260923 18:04:44.165347 5456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a: No bootstrap required, opened a new log
I20260923 18:04:44.168025 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:44.169516 5456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3178ca8a27bd4e4592b7bde0e12b095a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 43067 } }
I20260923 18:04:44.169734 5456 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:44.169811 5456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3178ca8a27bd4e4592b7bde0e12b095a, State: Initialized, Role: FOLLOWER
I20260923 18:04:44.170298 5456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3178ca8a27bd4e4592b7bde0e12b095a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 43067 } }
I20260923 18:04:44.170482 5456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:44.170562 5456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:44.170663 5456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:44.171645 5456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3178ca8a27bd4e4592b7bde0e12b095a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 43067 } }
I20260923 18:04:44.171985 5456 leader_election.cc:302] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3178ca8a27bd4e4592b7bde0e12b095a; no voters:
I20260923 18:04:44.172320 5456 leader_election.cc:288] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:44.172402 5466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:44.172611 5466 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [term 1 LEADER]: Becoming Leader. State: Replica: 3178ca8a27bd4e4592b7bde0e12b095a, State: Running, Role: LEADER
I20260923 18:04:44.172914 5466 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3178ca8a27bd4e4592b7bde0e12b095a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 43067 } }
I20260923 18:04:44.173646 5456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:44.174137 5466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3178ca8a27bd4e4592b7bde0e12b095a. Latest consensus state: current_term: 1 leader_uuid: "3178ca8a27bd4e4592b7bde0e12b095a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3178ca8a27bd4e4592b7bde0e12b095a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 43067 } } }
I20260923 18:04:44.174252 5466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:44.174573 5467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3178ca8a27bd4e4592b7bde0e12b095a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3178ca8a27bd4e4592b7bde0e12b095a" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 43067 } } }
I20260923 18:04:44.174710 5467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:44.176087 5471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:44.178838 5459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:44.170331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57449 (local address 127.15.139.190:43067)
0923 18:04:44.170440 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:44.170445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:44.170459 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:04:44.170495 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:44.170498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:44.170560 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:44.170654 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:44.171336 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:44.171926 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:44.172952 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:44.173265 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:44.173381 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:44.175256 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:44.175261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:44.175264 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:44.175290 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:44.177104 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:44.177554 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:44.177559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:44.177560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:44.177659 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:44.177964 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:44.177968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:44.177970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:44.178155 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:44.178291 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:44.178524 (+ 233us) server_negotiation.cc:299] Negotiation successful
0923 18:04:44.178658 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 18:04:44.179663 5471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:44.185061 5471 catalog_manager.cc:1347] Generated new cluster ID: f736ccf702cc41f69b8e51b2e74213db
I20260923 18:04:44.185129 5471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:44.196081 5471 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:44.197135 5471 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:44.205668 5471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3178ca8a27bd4e4592b7bde0e12b095a: Generated new TSK 0
I20260923 18:04:44.206485 5471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:44.213325 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 5388
2026-09-23T18:04:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:44 dist-test-slave-vr57 krb5kdc[5487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:44 dist-test-slave-vr57 krb5kdc[5487](info): set up 2 sockets
Sep 23 18:04:44 dist-test-slave-vr57 krb5kdc[5487](info): commencing operation
krb5kdc: starting...
W20260923 18:04:45.268373 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:45 dist-test-slave-vr57 krb5kdc[5487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:04:45.398322 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:39633
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:39785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:39633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:45.510932 5500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:45.511257 5500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:45.511340 5500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:45.515300 5500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:45.515398 5500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:45.515437 5500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:45.515477 5500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:45.515527 5500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:45.520666 5500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:39785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:39633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:39633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:45.521744 5500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:45.522805 5500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:45.532159 5508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:45.532163 5506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:45.532194 5505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:45.532508 5500 server_base.cc:1034] running on GCE node
I20260923 18:04:45.533188 5500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:45.534416 5500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:45.535650 5500 hybrid_clock.cc:648] HybridClock initialized: now 1790186685535632 us; error 38 us; skew 500 ppm
Sep 23 18:04:45 dist-test-slave-vr57 krb5kdc[5487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186685, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:45.538331 5500 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:04:45.539268 5500 webserver.cc:460] Webserver started at http://127.15.139.190:42159/ using document root <none> and password file <none>
I20260923 18:04:45.539922 5500 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:45.540019 5500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:45.540254 5500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:45.542294 5500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "531a7bf1fe0d4200a726140077385bf0"
format_stamp: "Formatted at 2026-09-23 18:04:45 on dist-test-slave-vr57"
server_key: "1377dec1401e2ed157acda58ebbad253"
server_key_iv: "80cab003d4eaceb0f9a4081077b44bce"
server_key_version: "encryptionkey@0"
I20260923 18:04:45.542874 5500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "531a7bf1fe0d4200a726140077385bf0"
format_stamp: "Formatted at 2026-09-23 18:04:45 on dist-test-slave-vr57"
server_key: "1377dec1401e2ed157acda58ebbad253"
server_key_iv: "80cab003d4eaceb0f9a4081077b44bce"
server_key_version: "encryptionkey@0"
I20260923 18:04:45.546764 5500 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:04:45.549340 5515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:45.550774 5500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:04:45.550963 5500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "531a7bf1fe0d4200a726140077385bf0"
format_stamp: "Formatted at 2026-09-23 18:04:45 on dist-test-slave-vr57"
server_key: "1377dec1401e2ed157acda58ebbad253"
server_key_iv: "80cab003d4eaceb0f9a4081077b44bce"
server_key_version: "encryptionkey@0"
I20260923 18:04:45.551091 5500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:45.574664 5500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:45.578845 5500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:45.579034 5500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:45.586383 5500 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:39633
I20260923 18:04:45.586449 5567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:39633 every 8 connection(s)
I20260923 18:04:45.587482 5500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:45.590370 5568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:45.595687 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 5500
I20260923 18:04:45.595834 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:45.596041 5568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0: Bootstrap starting.
I20260923 18:04:45.596163 15918 external_mini_cluster.cc:1428] Setting key 395df4eb6a3404fb7d86f072c190f879
I20260923 18:04:45.598729 5568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:45.599593 5568 log.cc:824] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:45.601755 5568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0: No bootstrap required, opened a new log
I20260923 18:04:45.605232 5571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:45.597637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56523 (local address 127.15.139.190:39633)
0923 18:04:45.598083 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:45.598099 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:45.598134 (+ 35us) server_negotiation.cc:407] Connection header received
0923 18:04:45.598759 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:45.598801 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:45.599166 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:45.599438 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:45.599881 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:45.600843 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:45.601999 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:45.602466 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:45.602587 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:45.603445 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:45.603450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:45.603462 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:45.603935 (+ 473us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:45.603988 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:45.604012 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:45.604093 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:45.604265 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 18:04:45.604487 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":162,"threads_started":1}
I20260923 18:04:45.605945 5568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "531a7bf1fe0d4200a726140077385bf0" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 39633 } }
I20260923 18:04:45.606163 5568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:45.606204 5568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 531a7bf1fe0d4200a726140077385bf0, State: Initialized, Role: FOLLOWER
I20260923 18:04:45.606666 5568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "531a7bf1fe0d4200a726140077385bf0" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 39633 } }
I20260923 18:04:45.606794 5568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:45.606876 5568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:45.606992 5568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:45.608134 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:45.608144 5568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "531a7bf1fe0d4200a726140077385bf0" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 39633 } }
I20260923 18:04:45.608486 5568 leader_election.cc:302] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 531a7bf1fe0d4200a726140077385bf0; no voters:
I20260923 18:04:45.608848 5568 leader_election.cc:288] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:45.608965 5573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:45.609179 5573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [term 1 LEADER]: Becoming Leader. State: Replica: 531a7bf1fe0d4200a726140077385bf0, State: Running, Role: LEADER
I20260923 18:04:45.609583 5573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "531a7bf1fe0d4200a726140077385bf0" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 39633 } }
I20260923 18:04:45.610140 5568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:45.610836 5575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 531a7bf1fe0d4200a726140077385bf0. Latest consensus state: current_term: 1 leader_uuid: "531a7bf1fe0d4200a726140077385bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "531a7bf1fe0d4200a726140077385bf0" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 39633 } } }
I20260923 18:04:45.611052 5575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:45.611227 5574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "531a7bf1fe0d4200a726140077385bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "531a7bf1fe0d4200a726140077385bf0" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 39633 } } }
I20260923 18:04:45.611340 5574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:45.611708 5587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:45.615439 5587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:45.615999 5571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:45.610352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50007 (local address 127.15.139.190:39633)
0923 18:04:45.610464 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:45.610471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:45.610877 (+ 406us) server_negotiation.cc:407] Connection header received
0923 18:04:45.611108 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:45.611114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:45.611183 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:45.611242 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:45.611798 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:45.612673 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:45.613859 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:45.614182 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:45.614289 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:45.615207 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:45.615210 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:45.615213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:45.615226 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:45.615254 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:45.615268 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:45.615348 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:45.615813 (+ 465us) server_negotiation.cc:299] Negotiation successful
0923 18:04:45.615867 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 18:04:45.620822 5587 catalog_manager.cc:1347] Generated new cluster ID: 777fe11241544f24bf172066cc67ffbf
I20260923 18:04:45.620889 5587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:45.664908 5587 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:45.666388 5587 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:45.673498 5587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 531a7bf1fe0d4200a726140077385bf0: Generated new TSK 0
I20260923 18:04:45.674443 5587 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:45.685657 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 5500
2026-09-23T18:04:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1471 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:45 dist-test-slave-vr57 krb5kdc[5599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:45 dist-test-slave-vr57 krb5kdc[5599](info): set up 2 sockets
Sep 23 18:04:45 dist-test-slave-vr57 krb5kdc[5599](info): commencing operation
krb5kdc: starting...
W20260923 18:04:46.740319 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:46 dist-test-slave-vr57 krb5kdc[5599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:04:46.866587 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:32771
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:32903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:32771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:46.980193 5612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:46.980520 5612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:46.980611 5612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:46.987106 5612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:46.987200 5612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:46.987252 5612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:46.987290 5612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:46.987323 5612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:46.992976 5612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:32903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:32771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:32771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:46.994215 5612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:46.995265 5612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:47.003975 5617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:47.004027 5612 server_base.cc:1034] running on GCE node
W20260923 18:04:47.003978 5620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:47.003976 5618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:47.004861 5612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:47.006313 5612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:47.007576 5612 hybrid_clock.cc:648] HybridClock initialized: now 1790186687007468 us; error 46 us; skew 500 ppm
Sep 23 18:04:47 dist-test-slave-vr57 krb5kdc[5599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186687, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:47.010517 5612 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:04:47.011569 5612 webserver.cc:460] Webserver started at http://127.15.139.190:34485/ using document root <none> and password file <none>
I20260923 18:04:47.012166 5612 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:47.012264 5612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:47.012506 5612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:47.014714 5612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "f9b9c648dc594370840bc48fe083974b"
format_stamp: "Formatted at 2026-09-23 18:04:47 on dist-test-slave-vr57"
server_key: "5f37a9dc6f114803ebce6d7da16164f4"
server_key_iv: "bafffebb1e06dc009c1aba5778beb446"
server_key_version: "encryptionkey@0"
I20260923 18:04:47.015303 5612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "f9b9c648dc594370840bc48fe083974b"
format_stamp: "Formatted at 2026-09-23 18:04:47 on dist-test-slave-vr57"
server_key: "5f37a9dc6f114803ebce6d7da16164f4"
server_key_iv: "bafffebb1e06dc009c1aba5778beb446"
server_key_version: "encryptionkey@0"
I20260923 18:04:47.019174 5612 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260923 18:04:47.021783 5627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:47.023077 5612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:04:47.023226 5612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "f9b9c648dc594370840bc48fe083974b"
format_stamp: "Formatted at 2026-09-23 18:04:47 on dist-test-slave-vr57"
server_key: "5f37a9dc6f114803ebce6d7da16164f4"
server_key_iv: "bafffebb1e06dc009c1aba5778beb446"
server_key_version: "encryptionkey@0"
I20260923 18:04:47.023336 5612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:47.028002 5612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:47.031248 5612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:47.031455 5612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:47.038736 5612 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:32771
I20260923 18:04:47.038825 5680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:32771 every 8 connection(s)
I20260923 18:04:47.039776 5612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:47.042568 5681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:47.042960 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 5612
I20260923 18:04:47.043119 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:47.043442 15918 external_mini_cluster.cc:1428] Setting key 751d83f6453b6229c1e447578b4b4ede
I20260923 18:04:47.049213 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:47.044962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42207 (local address 127.15.139.190:32771)
0923 18:04:47.045379 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:47.045398 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:47.045443 (+ 45us) server_negotiation.cc:407] Connection header received
0923 18:04:47.046201 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:47.046224 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:47.046597 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:47.046968 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:47.047788 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:47.047794 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:47.047809 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:47.047822 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:47.047897 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:47.047925 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:47.048083 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:47.048156 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 18:04:47.048305 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":107,"threads_started":1}
I20260923 18:04:47.049929 5681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b: Bootstrap starting.
I20260923 18:04:47.052248 5681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:47.052341 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:47.053133 5681 log.cc:824] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:47.055203 5681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b: No bootstrap required, opened a new log
I20260923 18:04:47.057389 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:47.055691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51745 (local address 127.15.139.190:32771)
0923 18:04:47.055812 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:47.055816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:47.056038 (+ 222us) server_negotiation.cc:407] Connection header received
0923 18:04:47.056129 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:47.056132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:47.056166 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:47.056246 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:47.056981 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:47.056984 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:47.056987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 18:04:47.056998 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:47.057034 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 18:04:47.057047 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:47.057135 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:47.057232 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 18:04:47.057268 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 18:04:47.059489 5681 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b9c648dc594370840bc48fe083974b" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 32771 } }
I20260923 18:04:47.059782 5681 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 18:04:47.059865 5681 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9b9c648dc594370840bc48fe083974b, State: Initialized, Role: FOLLOWER
I20260923 18:04:47.060348 5681 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b9c648dc594370840bc48fe083974b" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 32771 } }
I20260923 18:04:47.060501 5681 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 18:04:47.060595 5681 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 18:04:47.060693 5681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:47.061770 5681 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b9c648dc594370840bc48fe083974b" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 32771 } }
I20260923 18:04:47.062120 5681 leader_election.cc:302] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9b9c648dc594370840bc48fe083974b; no voters:
I20260923 18:04:47.062422 5681 leader_election.cc:288] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 18:04:47.062613 5691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:47.062847 5691 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [term 1 LEADER]: Becoming Leader. State: Replica: f9b9c648dc594370840bc48fe083974b, State: Running, Role: LEADER
I20260923 18:04:47.063174 5691 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b9c648dc594370840bc48fe083974b" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 32771 } }
I20260923 18:04:47.063748 5681 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:47.064965 5693 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9b9c648dc594370840bc48fe083974b. Latest consensus state: current_term: 1 leader_uuid: "f9b9c648dc594370840bc48fe083974b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b9c648dc594370840bc48fe083974b" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 32771 } } }
I20260923 18:04:47.065104 5693 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:47.065474 5700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:47.065429 5692 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9b9c648dc594370840bc48fe083974b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b9c648dc594370840bc48fe083974b" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 32771 } } }
I20260923 18:04:47.066102 5692 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:47.068305 5700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:47.073619 5700 catalog_manager.cc:1347] Generated new cluster ID: 7fd7284bcfd84e458d2670fed15f723e
I20260923 18:04:47.073685 5700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:47.092237 5700 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:47.092649 5700 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:47.096581 5700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9b9c648dc594370840bc48fe083974b: Generated new TSK 0
I20260923 18:04:47.097307 5700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 18:04:47.106441 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 5612
2026-09-23T18:04:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1420 ms)
[----------] 18 tests from AuthTokenIssuingTest (26046 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:47 dist-test-slave-vr57 krb5kdc[5712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:47 dist-test-slave-vr57 krb5kdc[5712](info): set up 2 sockets
Sep 23 18:04:47 dist-test-slave-vr57 krb5kdc[5712](info): commencing operation
krb5kdc: starting...
W20260923 18:04:48.172833 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:04:48.298763 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40653
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:38723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:40653,127.15.139.189:44061,127.15.139.188:39105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:48.410144 5725 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 18:04:48.410450 5725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:48.410475 5725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:48.410578 5725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:48.414662 5725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:48.414727 5725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:48.414747 5725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:48.414764 5725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:48.414809 5725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:48.420431 5725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:38723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:40653,127.15.139.189:44061,127.15.139.188:39105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:40653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:48.421617 5725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:48.422740 5725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:48.431685 5733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:48.431696 5731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:48.431685 5730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:48.432070 5725 server_base.cc:1034] running on GCE node
I20260923 18:04:48.432602 5725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:48.433759 5725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:48.434922 5725 hybrid_clock.cc:648] HybridClock initialized: now 1790186688434905 us; error 31 us; skew 500 ppm
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:48.444250 5725 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:04:48.445377 5725 webserver.cc:460] Webserver started at http://127.15.139.190:38521/ using document root <none> and password file <none>
I20260923 18:04:48.446061 5725 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:48.446162 5725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:48.446412 5725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:48.448637 5725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "b735f557abb24a9aab95d8be75ea8dee"
format_stamp: "Formatted at 2026-09-23 18:04:48 on dist-test-slave-vr57"
server_key: "45ef02b8ed81d04c499297d6da123ea9"
server_key_iv: "9de6c5269184a2b0c5532683a414f448"
server_key_version: "encryptionkey@0"
I20260923 18:04:48.449232 5725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "b735f557abb24a9aab95d8be75ea8dee"
format_stamp: "Formatted at 2026-09-23 18:04:48 on dist-test-slave-vr57"
server_key: "45ef02b8ed81d04c499297d6da123ea9"
server_key_iv: "9de6c5269184a2b0c5532683a414f448"
server_key_version: "encryptionkey@0"
I20260923 18:04:48.453197 5725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 18:04:48.455865 5740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:48.457194 5725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260923 18:04:48.457347 5725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "b735f557abb24a9aab95d8be75ea8dee"
format_stamp: "Formatted at 2026-09-23 18:04:48 on dist-test-slave-vr57"
server_key: "45ef02b8ed81d04c499297d6da123ea9"
server_key_iv: "9de6c5269184a2b0c5532683a414f448"
server_key_version: "encryptionkey@0"
I20260923 18:04:48.457466 5725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:48.482182 5725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:48.485774 5725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:48.485988 5725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:48.493508 5725 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:40653
I20260923 18:04:48.493819 5792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:40653 every 8 connection(s)
I20260923 18:04:48.494525 5725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:48.495702 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 5725
I20260923 18:04:48.495852 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:48.496145 15918 external_mini_cluster.cc:1428] Setting key 6fc52892c7abfa6663b8bdfcf0381483
I20260923 18:04:48.497758 5793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:48.503530 5793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.139.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.189@KRBTEST.COM" created.
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:48.520226 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.506430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40653 (local address 127.0.0.1:60574)
0923 18:04:48.506912 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.506962 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:48.507912 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:48.508775 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:48.508780 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:48.509109 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:48.509364 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.509377 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.510624 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.510627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.511348 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.511357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.512152 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.512156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.512320 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.512983 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:48.513000 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:48.514938 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:48.517128 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.517137 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.517146 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.517422 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.517820 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.517823 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.517824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.517940 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.518643 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:48.518649 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:48.518838 (+ 189us) client_negotiation.cc:769] Sending connection context
0923 18:04:48.518991 (+ 153us) client_negotiation.cc:240] Negotiation successful
0923 18:04:48.519347 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":131,"thread_start_us":199,"threads_started":1}
I20260923 18:04:48.520227 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.506344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60574 (local address 127.15.139.190:40653)
0923 18:04:48.506870 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:48.506885 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:48.507127 (+ 242us) server_negotiation.cc:407] Connection header received
0923 18:04:48.508265 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:48.508270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:48.508520 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:48.508719 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:48.509525 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.510482 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.511474 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.511997 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.512178 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.515099 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:48.515104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:48.515115 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:48.515138 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:48.517016 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.517596 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.517604 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.517606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.517723 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.518029 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.518032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.518034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.518471 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:48.518643 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:48.519094 (+ 451us) server_negotiation.cc:299] Negotiation successful
0923 18:04:48.519266 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":393,"thread_start_us":240,"threads_started":1}
I20260923 18:04:48.522698 5793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:04:48.523492 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.523223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44061 (local address 127.0.0.1:43860)
0923 18:04:48.523343 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.523430 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:44061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260923 18:04:48.523790 5743 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.139.189:44061: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.189.keytab.
Entry for principal kudu/127.15.139.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.189.keytab.
I20260923 18:04:48.524772 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.524557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44061 (local address 127.0.0.1:43868)
0923 18:04:48.524672 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.524719 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:44061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260923 18:04:48.524998 5793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.189:44061: Network error: Client connection negotiation failed: client connection to 127.15.139.189:44061: connect: Connection refused (error 111)
I20260923 18:04:48.525055 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.189:44061
--webserver_interface=127.15.139.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.189.keytab
--principal=kudu/127.15.139.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:38723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:40653,127.15.139.189:44061,127.15.139.188:39105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:04:48.574303 5793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } attempt: 1
I20260923 18:04:48.575356 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.575032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44061 (local address 127.0.0.1:43874)
0923 18:04:48.575206 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.575274 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:44061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260923 18:04:48.576164 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.575958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44061 (local address 127.0.0.1:43890)
0923 18:04:48.576057 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.576106 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:44061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 18:04:48.576319 5793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.189:44061: Network error: Client connection negotiation failed: client connection to 127.15.139.189:44061: connect: Connection refused (error 111)
W20260923 18:04:48.641016 5800 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 18:04:48.641350 5800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:48.641412 5800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:48.641496 5800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260923 18:04:48.644526 5793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } attempt: 2
I20260923 18:04:48.645484 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.645149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44061 (local address 127.0.0.1:43894)
0923 18:04:48.645333 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.645403 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:44061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260923 18:04:48.646188 5800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:48.646262 5800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:48.646283 5800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:48.646301 5800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:48.646354 5800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:48.646430 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.646117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44061 (local address 127.0.0.1:43896)
0923 18:04:48.646232 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.646321 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:44061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260923 18:04:48.646628 5793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.189:44061: Network error: Client connection negotiation failed: client connection to 127.15.139.189:44061: connect: Connection refused (error 111)
I20260923 18:04:48.652329 5800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:38723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:40653,127.15.139.189:44061,127.15.139.188:39105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.189.keytab
--principal=kudu/127.15.139.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.189:44061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.139.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:48.653475 5800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:48.654583 5800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:48.663561 5805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:48.663606 5806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:48.663585 5808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:48.663866 5800 server_base.cc:1034] running on GCE node
I20260923 18:04:48.664587 5800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:48.665808 5800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:48.667032 5800 hybrid_clock.cc:648] HybridClock initialized: now 1790186688667022 us; error 31 us; skew 500 ppm
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:48.670001 5800 init.cc:377] Logged in from keytab as kudu/127.15.139.189@KRBTEST.COM (short username kudu)
I20260923 18:04:48.671000 5800 webserver.cc:460] Webserver started at http://127.15.139.189:35679/ using document root <none> and password file <none>
I20260923 18:04:48.671657 5800 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:48.671754 5800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:48.672001 5800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:48.674093 5800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/data/instance:
uuid: "8b3d67e6cc3b427297ada3f235d392eb"
format_stamp: "Formatted at 2026-09-23 18:04:48 on dist-test-slave-vr57"
server_key: "27fce47bb2f8236b107cd473492d43f1"
server_key_iv: "79f0d7a5ea0c681e86fd7349e8857fa3"
server_key_version: "encryptionkey@0"
I20260923 18:04:48.674679 5800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/wal/instance:
uuid: "8b3d67e6cc3b427297ada3f235d392eb"
format_stamp: "Formatted at 2026-09-23 18:04:48 on dist-test-slave-vr57"
server_key: "27fce47bb2f8236b107cd473492d43f1"
server_key_iv: "79f0d7a5ea0c681e86fd7349e8857fa3"
server_key_version: "encryptionkey@0"
I20260923 18:04:48.678586 5800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:04:48.681226 5815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:48.682535 5800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:04:48.682680 5800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/wal
uuid: "8b3d67e6cc3b427297ada3f235d392eb"
format_stamp: "Formatted at 2026-09-23 18:04:48 on dist-test-slave-vr57"
server_key: "27fce47bb2f8236b107cd473492d43f1"
server_key_iv: "79f0d7a5ea0c681e86fd7349e8857fa3"
server_key_version: "encryptionkey@0"
I20260923 18:04:48.682830 5800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:48.701174 5800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:48.704563 5800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:48.704785 5800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:48.712563 5800 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.189:44061
I20260923 18:04:48.712961 5867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.189:44061 every 8 connection(s)
I20260923 18:04:48.713629 5800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
I20260923 18:04:48.716624 5868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:48.720644 5868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:04:48.722213 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 5800
I20260923 18:04:48.722347 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-1/wal/instance
I20260923 18:04:48.722676 15918 external_mini_cluster.cc:1428] Setting key 0dd6ce5198d209413a56fe59630769db
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.189@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.139.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.188@KRBTEST.COM" created.
I20260923 18:04:48.735832 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.722756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60580 (local address 127.15.139.190:40653)
0923 18:04:48.722903 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:48.722908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:48.723902 (+ 994us) server_negotiation.cc:407] Connection header received
0923 18:04:48.724881 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:48.724884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:48.724947 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:48.725057 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:48.726233 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.726835 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.728060 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.728311 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.728427 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.731333 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:48.731340 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:48.731346 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:48.731388 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:48.733250 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.734029 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.734033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.734035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.734200 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.734548 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.734554 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.734560 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.734847 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:48.734964 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:48.735560 (+ 596us) server_negotiation.cc:299] Negotiation successful
0923 18:04:48.735693 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 18:04:48.737083 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.723204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40653 (local address 127.0.0.1:60580)
0923 18:04:48.723727 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.723780 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:48.724664 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:48.725235 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:48.725241 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:48.725585 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:48.726079 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.726091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.726972 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.726976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.727932 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.727941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.728470 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.728474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.728640 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.729323 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:48.729339 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:48.731082 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:48.733373 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.733378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.733391 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.733912 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.734309 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.734311 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.734313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.734448 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.734964 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:48.734969 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:48.735224 (+ 255us) client_negotiation.cc:769] Sending connection context
0923 18:04:48.735439 (+ 215us) client_negotiation.cc:240] Negotiation successful
0923 18:04:48.735895 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":158,"threads_started":1}
I20260923 18:04:48.737797 5793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } attempt: 3
I20260923 18:04:48.738904 5868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.189@KRBTEST.COM for kudu/127.15.139.189@KRBTEST.COM
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.189@KRBTEST.COM
I20260923 18:04:48.750710 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.740322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44061 (local address 127.0.0.1:43910)
0923 18:04:48.740450 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.740464 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:48.740544 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:48.740942 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:48.740945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:48.741136 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:48.741265 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.741274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.742192 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.742199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.742809 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.742818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.743768 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.743771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.743876 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.744550 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:48.744570 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:48.746126 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:48.748729 (+ 2603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.748732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.748734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.749019 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.749388 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.749393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.749395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.749509 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.750100 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:48.750103 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:48.750209 (+ 106us) client_negotiation.cc:769] Sending connection context
0923 18:04:48.750346 (+ 137us) client_negotiation.cc:240] Negotiation successful
0923 18:04:48.750501 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260923 18:04:48.750871 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.740395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43910 (local address 127.15.139.189:44061)
0923 18:04:48.740694 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:48.740702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:48.740716 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:04:48.740753 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:48.740756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:48.740808 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:48.740903 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:48.741405 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.742059 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.742932 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.743604 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.743719 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.746337 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:48.746354 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:48.746374 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:48.746404 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:48.748616 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.749151 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.749157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.749158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.749267 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.749614 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.749620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.749622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.749970 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:48.750123 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:48.750455 (+ 332us) server_negotiation.cc:299] Negotiation successful
0923 18:04:48.750614 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":235,"thread_start_us":163,"threads_started":1}
I20260923 18:04:48.751528 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.738282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44061 (local address 127.0.0.1:43904)
0923 18:04:48.738410 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.738426 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:48.738507 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:48.739255 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:48.739259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:48.739456 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:48.739999 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.740007 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.741712 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.741717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.742680 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.742695 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.743782 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.743786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.743890 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.744597 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:48.744616 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:48.747001 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:48.749712 (+ 2711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.749715 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.749718 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.749930 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.750216 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.750219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.750221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.750308 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.750682 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:48.750686 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:48.750790 (+ 104us) client_negotiation.cc:769] Sending connection context
0923 18:04:48.751192 (+ 402us) client_negotiation.cc:240] Negotiation successful
0923 18:04:48.751342 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 18:04:48.751545 5873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.738405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43904 (local address 127.15.139.189:44061)
0923 18:04:48.738814 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:48.738826 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:48.738856 (+ 30us) server_negotiation.cc:407] Connection header received
0923 18:04:48.738896 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:48.738900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:48.739099 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:48.739248 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:48.740145 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.741554 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.742818 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.743604 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.743716 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.747229 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:48.747242 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:48.747244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:48.747265 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:48.749576 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.750033 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.750038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.750039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.750118 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.750409 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.750412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.750413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.750576 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:48.750690 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:48.751246 (+ 556us) server_negotiation.cc:299] Negotiation successful
0923 18:04:48.751356 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":300,"spinlock_wait_cycles":4480,"thread_start_us":212,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.188.keytab.
Entry for principal kudu/127.15.139.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.188.keytab.
I20260923 18:04:48.752988 5793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:04:48.753049 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.188:39105
--webserver_interface=127.15.139.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.188.keytab
--principal=kudu/127.15.139.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:38723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:40653,127.15.139.189:44061,127.15.139.188:39105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:04:48.753238 5868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:04:48.753985 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.753672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52126)
0923 18:04:48.753853 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.753928 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260923 18:04:48.754204 5816 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
I20260923 18:04:48.754809 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.753485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52120)
0923 18:04:48.754659 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.754727 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1091}
I20260923 18:04:48.755223 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.754929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52128)
0923 18:04:48.755088 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.755147 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260923 18:04:48.755375 5868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:39105: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
I20260923 18:04:48.755633 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.755265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52136)
0923 18:04:48.755431 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.755483 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260923 18:04:48.755759 5793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:39105: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
I20260923 18:04:48.786918 5793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } attempt: 1
I20260923 18:04:48.787952 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.787593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52142)
0923 18:04:48.787798 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.787870 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260923 18:04:48.788676 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.788454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52146)
0923 18:04:48.788563 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.788617 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260923 18:04:48.788812 5793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:39105: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
I20260923 18:04:48.804693 5868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } attempt: 1
I20260923 18:04:48.805689 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.805302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52148)
0923 18:04:48.805477 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.805553 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260923 18:04:48.806479 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.806240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52156)
0923 18:04:48.806353 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.806409 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260923 18:04:48.806648 5868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:39105: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
I20260923 18:04:48.863991 5793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } attempt: 2
I20260923 18:04:48.864837 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.864530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52160)
0923 18:04:48.864677 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.864745 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260923 18:04:48.865697 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.865472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52166)
0923 18:04:48.865586 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.865635 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260923 18:04:48.865875 5793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:39105: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
I20260923 18:04:48.874843 5868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } attempt: 2
I20260923 18:04:48.875738 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.875403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52168)
0923 18:04:48.875595 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.875658 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260923 18:04:48.876466 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.876252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52176)
0923 18:04:48.876360 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.876412 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 18:04:48.876616 5868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:39105: Network error: Client connection negotiation failed: client connection to 127.15.139.188:39105: connect: Connection refused (error 111)
W20260923 18:04:48.880402 5875 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 18:04:48.880770 5875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:48.880839 5875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:48.880952 5875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:48.885593 5875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:48.885689 5875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:48.885741 5875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:48.885804 5875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:48.885846 5875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:48.892125 5875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:38723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:40653,127.15.139.189:44061,127.15.139.188:39105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.188.keytab
--principal=kudu/127.15.139.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.188:39105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.139.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:48.893365 5875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:48.894526 5875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:48.904155 5881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:48.904172 5884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:48.904143 5882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:48.904542 5875 server_base.cc:1034] running on GCE node
I20260923 18:04:48.905170 5875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:48.906320 5875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:48.907492 5875 hybrid_clock.cc:648] HybridClock initialized: now 1790186688907457 us; error 48 us; skew 500 ppm
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:48.910461 5875 init.cc:377] Logged in from keytab as kudu/127.15.139.188@KRBTEST.COM (short username kudu)
I20260923 18:04:48.911415 5875 webserver.cc:460] Webserver started at http://127.15.139.188:46451/ using document root <none> and password file <none>
I20260923 18:04:48.912050 5875 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:48.912148 5875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:48.912398 5875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:48.914513 5875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/data/instance:
uuid: "8f7094dbf25240969fee40fb106242ec"
format_stamp: "Formatted at 2026-09-23 18:04:48 on dist-test-slave-vr57"
server_key: "1c087e77265c4f4539f68779aafb9734"
server_key_iv: "d99fb7486b9c58f044cdf3b3bc75be11"
server_key_version: "encryptionkey@0"
I20260923 18:04:48.915091 5875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/wal/instance:
uuid: "8f7094dbf25240969fee40fb106242ec"
format_stamp: "Formatted at 2026-09-23 18:04:48 on dist-test-slave-vr57"
server_key: "1c087e77265c4f4539f68779aafb9734"
server_key_iv: "d99fb7486b9c58f044cdf3b3bc75be11"
server_key_version: "encryptionkey@0"
I20260923 18:04:48.918926 5875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:04:48.921537 5891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:48.922806 5875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 18:04:48.922964 5875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/wal
uuid: "8f7094dbf25240969fee40fb106242ec"
format_stamp: "Formatted at 2026-09-23 18:04:48 on dist-test-slave-vr57"
server_key: "1c087e77265c4f4539f68779aafb9734"
server_key_iv: "d99fb7486b9c58f044cdf3b3bc75be11"
server_key_version: "encryptionkey@0"
I20260923 18:04:48.923089 5875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:48.934973 5875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:48.938302 5875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:48.938534 5875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:48.946189 5875 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.188:39105
I20260923 18:04:48.946480 5943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.188:39105 every 8 connection(s)
I20260923 18:04:48.947196 5875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
I20260923 18:04:48.950131 5944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:48.950603 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 5875
I20260923 18:04:48.950770 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790186398863621-15918-0/minicluster-data/master-2/wal/instance
I20260923 18:04:48.951040 15918 external_mini_cluster.cc:1428] Setting key 3622545d0c76656f13dcad5380d1bd1e
I20260923 18:04:48.954762 5944 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.188@KRBTEST.COM
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.188@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:48.965047 5793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } attempt: 3
I20260923 18:04:48.967612 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.957019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.15.139.190:40653)
0923 18:04:48.957181 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:48.957186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:48.957474 (+ 288us) server_negotiation.cc:407] Connection header received
0923 18:04:48.957625 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:48.957628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:48.957680 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:48.957767 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:48.958593 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.959264 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.960250 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.960634 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.960767 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.963750 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:48.963753 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:48.963755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:48.963780 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:48.965473 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.966021 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.966024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.966025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.966109 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.966432 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.966437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.966440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.966671 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:48.966744 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:48.967296 (+ 552us) server_negotiation.cc:299] Negotiation successful
0923 18:04:48.967417 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 18:04:48.968268 5868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } attempt: 3
I20260923 18:04:48.968590 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.952544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52184 (local address 127.15.139.188:39105)
0923 18:04:48.953006 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:48.953025 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:48.953105 (+ 80us) server_negotiation.cc:407] Connection header received
0923 18:04:48.953954 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:48.953976 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:48.954400 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:48.954763 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:48.955576 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.956932 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.958011 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.958714 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.958823 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.961972 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:48.961978 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:48.961988 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:48.962009 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:48.964490 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.964988 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.964996 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.965024 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.965164 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.965807 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.965811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.965814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.966385 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:48.966579 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:48.967028 (+ 449us) server_negotiation.cc:299] Negotiation successful
0923 18:04:48.967480 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":111,"threads_started":1}
I20260923 18:04:48.968816 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.956957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:40653 (local address 127.0.0.1:60596)
0923 18:04:48.957349 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.957388 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:48.957526 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:48.957814 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:48.957820 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:48.958293 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:48.958461 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.958469 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.959387 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.959392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.960105 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.960114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.960846 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.960851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.960971 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.961650 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:48.961671 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:48.963434 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:48.965594 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.965602 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.965609 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.965910 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.966231 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.966234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.966236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.966320 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.966829 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:48.966837 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:48.966991 (+ 154us) client_negotiation.cc:769] Sending connection context
0923 18:04:48.967744 (+ 753us) client_negotiation.cc:240] Negotiation successful
0923 18:04:48.967932 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":1,"thread_start_us":157,"threads_started":1}
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.188@KRBTEST.COM
I20260923 18:04:48.971690 5944 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.189@KRBTEST.COM for kudu/127.15.139.188@KRBTEST.COM
I20260923 18:04:48.975901 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.965517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52186)
0923 18:04:48.965648 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.965665 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:48.965749 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:48.966149 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:48.966152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:48.966308 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:48.966382 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.966386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.967314 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.967317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.968138 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.968149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.968725 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.968729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.968817 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.969279 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:48.969293 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:48.971188 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:48.974205 (+ 3017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.974211 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.974216 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.974477 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.974906 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.974910 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.974911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.974998 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.975412 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:48.975415 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:48.975495 (+ 80us) client_negotiation.cc:769] Sending connection context
0923 18:04:48.975594 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 18:04:48.975737 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 18:04:48.976790 5949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.965597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.15.139.188:39105)
0923 18:04:48.965916 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:48.965925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:48.965940 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:04:48.965975 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:48.965981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:48.966026 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:48.966108 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:48.966491 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.967135 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.968262 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.968571 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.971330 (+ 2759us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.971438 (+ 108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:48.971440 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:48.971442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:48.971455 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:48.974093 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.974613 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.974617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.974620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.974797 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.975108 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.975112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.975114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.975307 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:48.976392 (+ 1085us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:48.976471 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 18:04:48.976586 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":133,"threads_started":1}
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.188@KRBTEST.COM for kudu/127.15.139.189@KRBTEST.COM
I20260923 18:04:48.981164 5793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee: Bootstrap starting.
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:48.984198 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.968714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52190)
0923 18:04:48.968981 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.968997 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:48.969087 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:48.970041 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:48.970044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:48.970244 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:48.970372 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.970382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.971383 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.971387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.972199 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.972211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.972791 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.972795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.972881 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.973553 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:48.973571 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:48.975052 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:48.982222 (+ 7170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.982229 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.982233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.982559 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.982900 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.982903 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.982905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.983014 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.983562 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:48.983564 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:48.983682 (+ 118us) client_negotiation.cc:769] Sending connection context
0923 18:04:48.983800 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 18:04:48.983967 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189}
I20260923 18:04:48.984443 5793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:48.984730 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.968961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52190 (local address 127.15.139.188:39105)
0923 18:04:48.969787 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:48.969792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:48.969806 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:04:48.969844 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:48.969850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:48.969896 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:48.969975 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:48.970505 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.971266 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.972324 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.972667 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.972761 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.975204 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:48.975208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:48.975210 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:48.975233 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:48.982071 (+ 6838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.982686 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.982692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.982695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.982784 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.983119 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.983123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.983125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.983385 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:48.983485 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:48.984420 (+ 935us) server_negotiation.cc:299] Negotiation successful
0923 18:04:48.984554 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":105,"threads_started":1}
I20260923 18:04:48.985417 5793 log.cc:824] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:48.987753 5873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.972447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43924 (local address 127.15.139.189:44061)
0923 18:04:48.972560 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:48.972564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:48.972578 (+ 14us) server_negotiation.cc:407] Connection header received
0923 18:04:48.972911 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:48.972913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:48.972959 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:48.973046 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:48.973628 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.974421 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.975536 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.975962 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.976056 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.981712 (+ 5656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:48.981716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:48.981719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:48.982186 (+ 467us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:48.984700 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.986154 (+ 1454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.986159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.986164 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.986251 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.986553 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.986559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.986564 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.986888 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:48.986988 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:48.987409 (+ 421us) server_negotiation.cc:299] Negotiation successful
0923 18:04:48.987580 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 18:04:48.988204 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.972135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:44061 (local address 127.0.0.1:43924)
0923 18:04:48.972475 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.972489 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:48.972558 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:48.973160 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:48.973165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:48.973344 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:48.973486 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.973501 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.974566 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.974569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.975362 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.975371 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.976223 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.976229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.976352 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.977460 (+ 1108us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:48.977478 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:48.979538 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:48.985743 (+ 6205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.985747 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.985749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.986023 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.986380 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:48.986382 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:48.986383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:48.986453 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:48.987075 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:48.987078 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:48.987197 (+ 119us) client_negotiation.cc:769] Sending connection context
0923 18:04:48.987891 (+ 694us) client_negotiation.cc:240] Negotiation successful
0923 18:04:48.988045 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268}
I20260923 18:04:48.988554 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.973402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.15.139.190:40653)
0923 18:04:48.973527 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:48.973533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:48.973727 (+ 194us) server_negotiation.cc:407] Connection header received
0923 18:04:48.973910 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:48.973915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:48.973961 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:48.974014 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:48.974906 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.975636 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.976775 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.977177 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.977314 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.983080 (+ 5766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:48.983085 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:48.983090 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:48.983119 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:48.985558 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.987080 (+ 1522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.987083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.987085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.987172 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.987553 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.987557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.987558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.987787 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:48.987955 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:48.988339 (+ 384us) server_negotiation.cc:299] Negotiation successful
0923 18:04:48.988451 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 18:04:48.989921 5793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee: No bootstrap required, opened a new log
I20260923 18:04:48.990259 5868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb: Bootstrap starting.
I20260923 18:04:48.992044 5944 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:04:48.994131 5793 raft_consensus.cc:348] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } }
I20260923 18:04:48.994479 5793 raft_consensus.cc:729] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b735f557abb24a9aab95d8be75ea8dee, State: Initialized, Role: FOLLOWER
I20260923 18:04:48.994632 5868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:48.994977 5793 consensus_queue.cc:260] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } }
I20260923 18:04:48.995620 5868 log.cc:824] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:48.996367 5952 sys_catalog.cc:455] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } } }
I20260923 18:04:48.996541 5952 sys_catalog.cc:458] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:04:48.996600 5793 sys_catalog.cc:564] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [sys.catalog]: configured and running, proceeding with master startup.
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.189@KRBTEST.COM
Sep 23 18:04:48 dist-test-slave-vr57 krb5kdc[5712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186688, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.188@KRBTEST.COM for kudu/127.15.139.188@KRBTEST.COM
I20260923 18:04:48.999541 5868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb: No bootstrap required, opened a new log
I20260923 18:04:49.001453 5873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.990381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43940 (local address 127.15.139.189:44061)
0923 18:04:48.990523 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:48.990529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:48.990593 (+ 64us) server_negotiation.cc:407] Connection header received
0923 18:04:48.990887 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:48.990890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:48.990974 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:48.991044 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:48.992004 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.992744 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.993817 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.994136 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.994208 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.997366 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:48.997373 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:48.997378 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:48.997409 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:48.999094 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:48.999632 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:48.999635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:48.999637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:48.999720 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:49.000052 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:49.000055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:49.000057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:49.000278 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:49.000440 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:49.001159 (+ 719us) server_negotiation.cc:299] Negotiation successful
0923 18:04:49.001290 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 18:04:49.002640 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.992549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:39105 (local address 127.0.0.1:52202)
0923 18:04:48.992645 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:48.992658 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:48.992732 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:48.993013 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:48.993016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:48.993228 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:48.993318 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.993325 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.994274 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.994278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.995059 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:48.995073 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.995870 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.995875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:48.995949 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.996899 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:48.996913 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:48.999113 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:49.001134 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:49.001136 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:49.001138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:49.001376 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:49.001641 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:49.001643 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:49.001645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:49.001730 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:49.002167 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:49.002170 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:49.002255 (+ 85us) client_negotiation.cc:769] Sending connection context
0923 18:04:49.002347 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 18:04:49.002475 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 18:04:49.003162 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:48.992713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.15.139.188:39105)
0923 18:04:48.992805 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:48.992809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:48.992822 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:04:48.992856 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:48.992860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:48.992900 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:48.992983 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:48.993430 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.994115 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.995205 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:48.995621 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:48.995713 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:48.999299 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:48.999303 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:48.999305 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:48.999329 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:49.001028 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:49.001477 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:49.001480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:49.001482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:49.001554 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:49.001823 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:49.001826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:49.001828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:49.002027 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:49.002123 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:49.002911 (+ 788us) server_negotiation.cc:299] Negotiation successful
0923 18:04:49.003017 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 18:04:49.004555 5868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } }
I20260923 18:04:49.004787 5868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b3d67e6cc3b427297ada3f235d392eb, State: Initialized, Role: FOLLOWER
I20260923 18:04:49.005110 5868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } }
I20260923 18:04:49.005568 5965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } } }
I20260923 18:04:49.005743 5965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:04:49.002072 5725 master_runner.cc:186] Error getting master registration for 127.15.139.189:44061: OK, instance_id { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" instance_seqno: 1790186688710355 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 18:04:49.006026 5725 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 18:04:49.006168 5944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec: Bootstrap starting.
I20260923 18:04:49.006932 5868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:49.007354 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:49.008941 5944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:49.008848 5767 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b735f557abb24a9aab95d8be75ea8dee"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60602
I20260923 18:04:49.009090 5767 raft_consensus.cc:482] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 18:04:49.009219 5767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [term 0 FOLLOWER]: Advancing to term 1
W20260923 18:04:49.009514 5964 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:04:49.009574 5964 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:04:49.009806 5944 log.cc:824] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:49.010612 5767 raft_consensus.cc:504] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } }
I20260923 18:04:49.011633 5767 leader_election.cc:288] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [CANDIDATE]: Term 1 election: Requested vote from peers 8b3d67e6cc3b427297ada3f235d392eb (127.15.139.189:44061), 8f7094dbf25240969fee40fb106242ec (127.15.139.188:39105)
I20260923 18:04:49.012328 5842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b735f557abb24a9aab95d8be75ea8dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8b3d67e6cc3b427297ada3f235d392eb"
I20260923 18:04:49.012547 5842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [term 0 FOLLOWER]: Advancing to term 1
W20260923 18:04:49.012828 5977 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:04:49.012957 5977 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:04:49.013778 5944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec: No bootstrap required, opened a new log
I20260923 18:04:49.013847 5842 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b735f557abb24a9aab95d8be75ea8dee in term 1.
I20260923 18:04:49.014309 5743 leader_election.cc:302] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b3d67e6cc3b427297ada3f235d392eb, b735f557abb24a9aab95d8be75ea8dee; no voters:
I20260923 18:04:49.014675 5952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:49.014942 5952 raft_consensus.cc:686] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [term 1 LEADER]: Becoming Leader. State: Replica: b735f557abb24a9aab95d8be75ea8dee, State: Running, Role: LEADER
I20260923 18:04:49.015326 5952 consensus_queue.cc:237] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } }
I20260923 18:04:49.017262 5952 sys_catalog.cc:455] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [sys.catalog]: SysCatalogTable state changed. Reason: New leader b735f557abb24a9aab95d8be75ea8dee. Latest consensus state: current_term: 1 leader_uuid: "b735f557abb24a9aab95d8be75ea8dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } } }
I20260923 18:04:49.017436 5952 sys_catalog.cc:458] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:49.017854 5986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:49.017943 5944 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } }
I20260923 18:04:49.018253 5944 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7094dbf25240969fee40fb106242ec, State: Initialized, Role: FOLLOWER
I20260923 18:04:49.018646 5944 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } }
I20260923 18:04:49.018886 5986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:49.020089 5944 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:49.017663 5800 master_runner.cc:186] Error getting master registration for 127.15.139.188:39105: OK, instance_id { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" instance_seqno: 1790186688944130 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 18:04:49.021185 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b735f557abb24a9aab95d8be75ea8dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8f7094dbf25240969fee40fb106242ec"
I20260923 18:04:49.021428 5918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:49.023415 5918 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b735f557abb24a9aab95d8be75ea8dee in term 1.
I20260923 18:04:49.024091 5800 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 18:04:49.024282 5987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } } }
I20260923 18:04:49.024613 5987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [sys.catalog]: This master's current role is: FOLLOWER
W20260923 18:04:49.027590 5999 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:04:49.027711 5999 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:04:49.030514 5842 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [term 1 FOLLOWER]: Refusing update from remote peer b735f557abb24a9aab95d8be75ea8dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:49.031689 5918 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [term 1 FOLLOWER]: Refusing update from remote peer b735f557abb24a9aab95d8be75ea8dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:49.034327 5952 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:49.034793 5952 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:49.037817 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:49.024843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52216 (local address 127.15.139.188:39105)
0923 18:04:49.024966 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.024972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.026987 (+ 2015us) server_negotiation.cc:407] Connection header received
0923 18:04:49.027183 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.027186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.027230 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.027301 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.028438 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:49.029474 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:49.030872 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:49.031239 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:49.031336 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:49.033539 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:49.033544 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:49.033547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:49.033576 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:49.035674 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:49.036237 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:49.036242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:49.036244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:49.036334 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:49.036717 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:49.036721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:49.036723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:49.036954 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:49.037068 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:49.037481 (+ 413us) server_negotiation.cc:299] Negotiation successful
0923 18:04:49.037631 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 18:04:49.039144 5987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader b735f557abb24a9aab95d8be75ea8dee. Latest consensus state: current_term: 1 leader_uuid: "b735f557abb24a9aab95d8be75ea8dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } } }
I20260923 18:04:49.039275 5987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:04:49.041519 5952 sys_catalog.cc:455] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b735f557abb24a9aab95d8be75ea8dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } } }
I20260923 18:04:49.041661 5952 sys_catalog.cc:458] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:49.041800 5989 mvcc.cc:205] Tried to move back new op lower bound from 7332604678244757504 to 7332604678210863104. Current Snapshot: MvccSnapshot[applied={T|T < 7332604678244757504}]
I20260923 18:04:49.041915 5965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b735f557abb24a9aab95d8be75ea8dee. Latest consensus state: current_term: 1 leader_uuid: "b735f557abb24a9aab95d8be75ea8dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } } }
I20260923 18:04:49.042138 5965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:04:49.042837 5987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b735f557abb24a9aab95d8be75ea8dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } } }
I20260923 18:04:49.042980 5987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:04:49.048237 6004 mvcc.cc:205] Tried to move back new op lower bound from 7332604678244757504 to 7332604678210863104. Current Snapshot: MvccSnapshot[applied={T|T < 7332604678244757504}]
I20260923 18:04:49.048554 5986 catalog_manager.cc:1347] Generated new cluster ID: 4248c248fc8f4dee9eabef78e1cab61c
I20260923 18:04:49.048619 5986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:49.050014 5873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:49.025649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43944 (local address 127.15.139.189:44061)
0923 18:04:49.025769 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.025774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.027522 (+ 1748us) server_negotiation.cc:407] Connection header received
0923 18:04:49.027590 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.027593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.027642 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.027716 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.035606 (+ 7890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:49.036342 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:49.043589 (+ 7247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:49.043829 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:49.043890 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:49.045674 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:49.045678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:49.045681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:49.045709 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:49.047778 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:49.048302 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:49.048309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:49.048314 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:49.048417 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:49.049012 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:49.049016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:49.049017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:49.049245 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:49.049347 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:49.049729 (+ 382us) server_negotiation.cc:299] Negotiation successful
0923 18:04:49.049848 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 18:04:49.052177 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:49.023964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60604 (local address 127.15.139.190:40653)
0923 18:04:49.024091 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.024096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.026006 (+ 1910us) server_negotiation.cc:407] Connection header received
0923 18:04:49.026239 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.026242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.026289 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.026342 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.043566 (+ 17224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:49.044537 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:49.045820 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:49.046212 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:49.046300 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:49.048160 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:49.048164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:49.048168 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:49.048196 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:49.050078 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:49.050659 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:49.050663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:49.050665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:49.050756 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:49.051081 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:49.051085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:49.051087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:49.051302 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:49.051417 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:49.051878 (+ 461us) server_negotiation.cc:299] Negotiation successful
0923 18:04:49.052005 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 18:04:49.068104 5952 sys_catalog.cc:455] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b735f557abb24a9aab95d8be75ea8dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } } }
I20260923 18:04:49.068269 5952 sys_catalog.cc:458] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:49.069732 5965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b735f557abb24a9aab95d8be75ea8dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b735f557abb24a9aab95d8be75ea8dee" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 40653 } } peers { permanent_uuid: "8b3d67e6cc3b427297ada3f235d392eb" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 44061 } } peers { permanent_uuid: "8f7094dbf25240969fee40fb106242ec" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 39105 } } }
I20260923 18:04:49.069878 5965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:04:49.070472 6006 mvcc.cc:205] Tried to move back new op lower bound from 7332604678244757504 to 7332604678210863104. Current Snapshot: MvccSnapshot[applied={T|T < 7332604678244757504}]
I20260923 18:04:49.100347 5986 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:49.101979 5986 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:49.123958 5986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b735f557abb24a9aab95d8be75ea8dee: Generated new TSK 0
I20260923 18:04:49.125121 5986 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 18:04:49.157871 5873 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.154201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.15.139.189:44061)
0923 18:04:49.154350 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.154357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.156541 (+ 2184us) server_negotiation.cc:407] Connection header received
0923 18:04:49.156787 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.156791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.156855 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.156920 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.157707 (+ 787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43948: BlockingRecv error: recv got EOF from 127.0.0.1:43948 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260923 18:04:49.159812 5797 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.155126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60612 (local address 127.15.139.190:40653)
0923 18:04:49.155262 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.155267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.158663 (+ 3396us) server_negotiation.cc:407] Connection header received
0923 18:04:49.158875 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.158879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.158940 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.159004 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.159671 (+ 667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60612: BlockingRecv error: recv got EOF from 127.0.0.1:60612 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260923 18:04:49.161754 5947 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.155927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52218 (local address 127.15.139.188:39105)
0923 18:04:49.156088 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.156093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.160562 (+ 4469us) server_negotiation.cc:407] Connection header received
0923 18:04:49.160765 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.160769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.160845 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.160908 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.161618 (+ 710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52218: BlockingRecv error: recv got EOF from 127.0.0.1:52218 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260923 18:04:49.167596 5873 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.165801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43958 (local address 127.15.139.189:44061)
0923 18:04:49.165945 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.165953 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.166266 (+ 313us) server_negotiation.cc:407] Connection header received
0923 18:04:49.166556 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.166560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.166656 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.166720 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.167370 (+ 650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43958: BlockingRecv error: recv got EOF from 127.0.0.1:43958 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260923 18:04:49.173568 5873 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.171719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43966 (local address 127.15.139.189:44061)
0923 18:04:49.171837 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.171842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.172426 (+ 584us) server_negotiation.cc:407] Connection header received
0923 18:04:49.172676 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.172680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.172739 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.172800 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.173437 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43966: BlockingRecv error: recv got EOF from 127.0.0.1:43966 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260923 18:04:49.180094 5873 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.178170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43970 (local address 127.15.139.189:44061)
0923 18:04:49.178289 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.178295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.178938 (+ 643us) server_negotiation.cc:407] Connection header received
0923 18:04:49.179171 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.179175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.179237 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.179299 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.179968 (+ 669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43970: BlockingRecv error: recv got EOF from 127.0.0.1:43970 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260923 18:04:49.189020 5873 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.186837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43972 (local address 127.15.139.189:44061)
0923 18:04:49.187006 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.187012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.187846 (+ 834us) server_negotiation.cc:407] Connection header received
0923 18:04:49.188088 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.188092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.188155 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.188233 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.188886 (+ 653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43972: BlockingRecv error: recv got EOF from 127.0.0.1:43972 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260923 18:04:49.201897 5873 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.199885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43988 (local address 127.15.139.189:44061)
0923 18:04:49.200027 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.200032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.200759 (+ 727us) server_negotiation.cc:407] Connection header received
0923 18:04:49.200992 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.200996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.201072 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.201132 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.201764 (+ 632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43988: BlockingRecv error: recv got EOF from 127.0.0.1:43988 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260923 18:04:49.222551 5873 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.220741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.15.139.189:44061)
0923 18:04:49.220868 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.220873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.221483 (+ 610us) server_negotiation.cc:407] Connection header received
0923 18:04:49.221708 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.221712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.221767 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.221821 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.222423 (+ 602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44000: BlockingRecv error: recv got EOF from 127.0.0.1:44000 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260923 18:04:49.258517 5873 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.257200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44016 (local address 127.15.139.189:44061)
0923 18:04:49.257334 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.257339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.257602 (+ 263us) server_negotiation.cc:407] Connection header received
0923 18:04:49.257772 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.257780 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.257833 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.257929 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.258384 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44016: BlockingRecv error: recv got EOF from 127.0.0.1:44016 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260923 18:04:49.326081 5873 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.324741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44032 (local address 127.15.139.189:44061)
0923 18:04:49.324890 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.324895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.325092 (+ 197us) server_negotiation.cc:407] Connection header received
0923 18:04:49.325271 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.325279 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.325330 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.325426 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.325933 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44032: BlockingRecv error: recv got EOF from 127.0.0.1:44032 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260923 18:04:49.457556 5873 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.456412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44042 (local address 127.15.139.189:44061)
0923 18:04:49.456548 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.456552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.456692 (+ 140us) server_negotiation.cc:407] Connection header received
0923 18:04:49.456811 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.456813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.456862 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.456944 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.457440 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44042: BlockingRecv error: recv got EOF from 127.0.0.1:44042 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260923 18:04:49.716924 5873 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:49.715837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44044 (local address 127.15.139.189:44061)
0923 18:04:49.715981 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:49.715986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:49.716067 (+ 81us) server_negotiation.cc:407] Connection header received
0923 18:04:49.716200 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:49.716203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:49.716260 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:49.716356 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:49.716806 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44044: BlockingRecv error: recv got EOF from 127.0.0.1:44044 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 18:04:50.014351 5977 catalog_manager.cc:1259] Loaded cluster ID: 4248c248fc8f4dee9eabef78e1cab61c
I20260923 18:04:50.014477 5977 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb: loading cluster ID for follower catalog manager: success
I20260923 18:04:50.016219 5977 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb: acquiring CA information for follower catalog manager: success
I20260923 18:04:50.017402 5977 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8b3d67e6cc3b427297ada3f235d392eb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 18:04:50.029129 5999 catalog_manager.cc:1259] Loaded cluster ID: 4248c248fc8f4dee9eabef78e1cab61c
I20260923 18:04:50.029211 5999 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec: loading cluster ID for follower catalog manager: success
I20260923 18:04:50.031172 5999 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec: acquiring CA information for follower catalog manager: success
I20260923 18:04:50.032508 5999 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8f7094dbf25240969fee40fb106242ec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 18:04:50.235486 6076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:50.231299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44050 (local address 127.15.139.189:44061)
0923 18:04:50.231634 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:50.231637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:50.231657 (+ 20us) server_negotiation.cc:407] Connection header received
0923 18:04:50.231710 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:50.231712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:50.231848 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:50.231950 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:50.232245 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:50.232835 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:50.233725 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:50.233953 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:50.234036 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:50.234396 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:50.234928 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:50.235041 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:50.235195 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 18:04:50.235297 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":79,"threads_started":1}
I20260923 18:04:50.243170 6082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:50.239239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52228 (local address 127.15.139.188:39105)
0923 18:04:50.239487 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:50.239493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:50.239536 (+ 43us) server_negotiation.cc:407] Connection header received
0923 18:04:50.239726 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:50.239729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:50.239852 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:50.239987 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:50.240229 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:50.240774 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:50.241644 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:50.241891 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:50.242003 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:50.242321 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:50.242762 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:50.242907 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:50.242944 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 18:04:50.243002 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":64,"threads_started":1}
I20260923 18:04:50.250836 6082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:50.246786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.15.139.188:39105)
0923 18:04:50.246885 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:50.246888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:50.246899 (+ 11us) server_negotiation.cc:407] Connection header received
0923 18:04:50.246938 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:50.246940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:50.246983 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:50.247064 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:50.247332 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:50.248143 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:50.249539 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:50.249884 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:50.250005 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:50.250281 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:50.250347 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:50.250427 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:50.250518 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 18:04:50.250656 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 18:04:50.251078 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:50.246202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60622 (local address 127.15.139.190:40653)
0923 18:04:50.246534 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:50.246538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:50.246564 (+ 26us) server_negotiation.cc:407] Connection header received
0923 18:04:50.246799 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:50.246804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:50.246978 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:50.247092 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:50.247553 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:50.248329 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:50.249243 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:50.249512 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:50.249620 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:50.249940 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:50.250373 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:50.250466 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:50.250845 (+ 379us) server_negotiation.cc:299] Negotiation successful
0923 18:04:50.250918 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":73,"threads_started":1}
I20260923 18:04:50.251811 6076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:50.246295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44056 (local address 127.15.139.189:44061)
0923 18:04:50.246438 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:50.246443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:50.246681 (+ 238us) server_negotiation.cc:407] Connection header received
0923 18:04:50.246892 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:50.246895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:50.246950 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:50.247001 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:50.247522 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:50.248158 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:50.250080 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:50.250391 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:50.250494 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:50.251224 (+ 730us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:50.251308 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:50.251378 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:50.251556 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 18:04:50.251620 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 18:04:50.254865 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 5725
I20260923 18:04:50.261327 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 5800
I20260923 18:04:50.267654 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 5875
2026-09-23T18:04:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3163 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 18:04:50 dist-test-slave-vr57 krb5kdc[6093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 18:04:50 dist-test-slave-vr57 krb5kdc[6093](info): set up 2 sockets
Sep 23 18:04:50 dist-test-slave-vr57 krb5kdc[6093](info): commencing operation
krb5kdc: starting...
W20260923 18:04:51.320353 15918 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/test-user.keytab.
Sep 23 18:04:51 dist-test-slave-vr57 krb5kdc[6093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T18:04:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T18:04:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.139.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
Entry for principal kudu/127.15.139.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab.
I20260923 18:04:51.446547 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:34101
--webserver_interface=127.15.139.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:35361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:34101,127.15.139.189:37283,127.15.139.188:33745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 18:04:51.558995 6106 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 18:04:51.559275 6106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:51.559333 6106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:51.559413 6106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:51.563439 6106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:51.563552 6106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:51.563591 6106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:51.563629 6106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:51.563654 6106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:51.569038 6106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:35361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:34101,127.15.139.189:37283,127.15.139.188:33745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.190.keytab
--principal=kudu/127.15.139.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.190:34101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.139.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:51.570197 6106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:51.571260 6106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:51.579967 6112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:51.579962 6111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:51.579962 6114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:51.580156 6106 server_base.cc:1034] running on GCE node
I20260923 18:04:51.580862 6106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:51.582012 6106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:51.583228 6106 hybrid_clock.cc:648] HybridClock initialized: now 1790186691583206 us; error 35 us; skew 500 ppm
Sep 23 18:04:51 dist-test-slave-vr57 krb5kdc[6093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186691, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:51.586265 6106 init.cc:377] Logged in from keytab as kudu/127.15.139.190@KRBTEST.COM (short username kudu)
I20260923 18:04:51.587199 6106 webserver.cc:460] Webserver started at http://127.15.139.190:41291/ using document root <none> and password file <none>
I20260923 18:04:51.587872 6106 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:51.587967 6106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:51.588196 6106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:51.590215 6106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/data/instance:
uuid: "31542cf98e9544768c0936b332f15be3"
format_stamp: "Formatted at 2026-09-23 18:04:51 on dist-test-slave-vr57"
server_key: "54da3273304834ed6ceb6d2242577171"
server_key_iv: "c8beef5cc15800efa421a4d79f18a28b"
server_key_version: "encryptionkey@0"
I20260923 18:04:51.590780 6106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/wal/instance:
uuid: "31542cf98e9544768c0936b332f15be3"
format_stamp: "Formatted at 2026-09-23 18:04:51 on dist-test-slave-vr57"
server_key: "54da3273304834ed6ceb6d2242577171"
server_key_iv: "c8beef5cc15800efa421a4d79f18a28b"
server_key_version: "encryptionkey@0"
I20260923 18:04:51.594516 6106 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260923 18:04:51.597121 6121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:51.598372 6106 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:04:51.598522 6106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/wal
uuid: "31542cf98e9544768c0936b332f15be3"
format_stamp: "Formatted at 2026-09-23 18:04:51 on dist-test-slave-vr57"
server_key: "54da3273304834ed6ceb6d2242577171"
server_key_iv: "c8beef5cc15800efa421a4d79f18a28b"
server_key_version: "encryptionkey@0"
I20260923 18:04:51.598636 6106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:51.616456 6106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:51.619856 6106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:51.620074 6106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:51.627604 6106 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.190:34101
I20260923 18:04:51.627897 6173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.190:34101 every 8 connection(s)
I20260923 18:04:51.628705 6106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/data/info.pb
I20260923 18:04:51.631626 6174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:51.633109 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 6106
I20260923 18:04:51.633251 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-0/wal/instance
I20260923 18:04:51.633581 15918 external_mini_cluster.cc:1428] Setting key 7ef018591a621ec746c14708687d5b5b
I20260923 18:04:51.636658 6174 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.139.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.189@KRBTEST.COM" created.
Sep 23 18:04:51 dist-test-slave-vr57 krb5kdc[6093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186691, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:51.653468 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.639171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:34101 (local address 127.0.0.1:35346)
0923 18:04:51.639778 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.639814 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:51.640637 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:51.641345 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:51.641350 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:51.641784 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:51.642022 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:51.642035 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.643368 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.643371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:51.644420 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:51.644436 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.645211 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.645218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:51.645365 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:51.646029 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:51.646045 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:51.647643 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:51.650325 (+ 2682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:51.650333 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:51.650343 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:51.650580 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:51.651030 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:51.651032 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:51.651034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:51.651158 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:51.652038 (+ 880us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:51.652044 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:51.652185 (+ 141us) client_negotiation.cc:769] Sending connection context
0923 18:04:51.652331 (+ 146us) client_negotiation.cc:240] Negotiation successful
0923 18:04:51.652621 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":158,"threads_started":1}
I20260923 18:04:51.653472 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.639164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35346 (local address 127.15.139.190:34101)
0923 18:04:51.639635 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:51.639654 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:51.639924 (+ 270us) server_negotiation.cc:407] Connection header received
0923 18:04:51.640949 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:51.640953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:51.641146 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:51.641351 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:51.642179 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.643225 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.644611 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.645031 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.645165 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:51.647808 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:51.647818 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:51.647836 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:51.647868 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:51.650186 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:51.650762 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:51.650771 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:51.650773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:51.650927 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:51.651273 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:51.651276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:51.651278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:51.651907 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:51.652060 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:51.652478 (+ 418us) server_negotiation.cc:299] Negotiation successful
0923 18:04:51.652755 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":283,"thread_start_us":134,"threads_started":1}
I20260923 18:04:51.656047 6174 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:04:51.656847 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.656517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:37283 (local address 127.0.0.1:34860)
0923 18:04:51.656714 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.656782 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:37283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260923 18:04:51.657071 6124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.139.189:37283: connect: Connection refused (error 111)
I20260923 18:04:51.658094 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.657878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:37283 (local address 127.0.0.1:34874)
0923 18:04:51.657979 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.658027 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:37283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260923 18:04:51.658296 6174 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.189:37283: Network error: Client connection negotiation failed: client connection to 127.15.139.189:37283: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.189.keytab.
Entry for principal kudu/127.15.139.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.189.keytab.
I20260923 18:04:51.663345 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.189:37283
--webserver_interface=127.15.139.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.189.keytab
--principal=kudu/127.15.139.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:35361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:34101,127.15.139.189:37283,127.15.139.188:33745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:04:51.707612 6174 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } attempt: 1
I20260923 18:04:51.708648 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.708277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:37283 (local address 127.0.0.1:34876)
0923 18:04:51.708442 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.708509 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:37283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260923 18:04:51.709509 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.709263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:37283 (local address 127.0.0.1:34884)
0923 18:04:51.709386 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.709446 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:37283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260923 18:04:51.709666 6174 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.189:37283: Network error: Client connection negotiation failed: client connection to 127.15.139.189:37283: connect: Connection refused (error 111)
I20260923 18:04:51.777853 6174 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } attempt: 2
I20260923 18:04:51.778784 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.778447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:37283 (local address 127.0.0.1:34896)
0923 18:04:51.778636 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.778703 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:37283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260923 18:04:51.778760 6181 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 18:04:51.779162 6181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:51.779233 6181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:51.779321 6181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260923 18:04:51.779734 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.779468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:37283 (local address 127.0.0.1:34898)
0923 18:04:51.779621 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.779678 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.189:37283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260923 18:04:51.779886 6174 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.189:37283: Network error: Client connection negotiation failed: client connection to 127.15.139.189:37283: connect: Connection refused (error 111)
W20260923 18:04:51.783716 6181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:51.783773 6181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:51.783797 6181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:51.783855 6181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:51.783895 6181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:51.789304 6181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:35361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:34101,127.15.139.189:37283,127.15.139.188:33745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.189.keytab
--principal=kudu/127.15.139.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.189:37283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.139.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:51.790349 6181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:51.791373 6181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:51.800081 6186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:51.800189 6189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:51.800081 6187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:51.800436 6181 server_base.cc:1034] running on GCE node
I20260923 18:04:51.800993 6181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:51.802193 6181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:51.803392 6181 hybrid_clock.cc:648] HybridClock initialized: now 1790186691803376 us; error 37 us; skew 500 ppm
Sep 23 18:04:51 dist-test-slave-vr57 krb5kdc[6093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186691, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:51.806144 6181 init.cc:377] Logged in from keytab as kudu/127.15.139.189@KRBTEST.COM (short username kudu)
I20260923 18:04:51.807061 6181 webserver.cc:460] Webserver started at http://127.15.139.189:35591/ using document root <none> and password file <none>
I20260923 18:04:51.807711 6181 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:51.807803 6181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:51.808051 6181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:51.810087 6181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/data/instance:
uuid: "003312950eea4fedab80fc91ee9a3d3c"
format_stamp: "Formatted at 2026-09-23 18:04:51 on dist-test-slave-vr57"
server_key: "60b9f175dbe240868a6db7d0f6c50e8b"
server_key_iv: "00af7368895e019e552ed6a50b935195"
server_key_version: "encryptionkey@0"
I20260923 18:04:51.810667 6181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/wal/instance:
uuid: "003312950eea4fedab80fc91ee9a3d3c"
format_stamp: "Formatted at 2026-09-23 18:04:51 on dist-test-slave-vr57"
server_key: "60b9f175dbe240868a6db7d0f6c50e8b"
server_key_iv: "00af7368895e019e552ed6a50b935195"
server_key_version: "encryptionkey@0"
I20260923 18:04:51.814512 6181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260923 18:04:51.817060 6196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:51.818320 6181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 18:04:51.818468 6181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/wal
uuid: "003312950eea4fedab80fc91ee9a3d3c"
format_stamp: "Formatted at 2026-09-23 18:04:51 on dist-test-slave-vr57"
server_key: "60b9f175dbe240868a6db7d0f6c50e8b"
server_key_iv: "00af7368895e019e552ed6a50b935195"
server_key_version: "encryptionkey@0"
I20260923 18:04:51.818585 6181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:51.838210 6181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:51.841498 6181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:51.841715 6181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:51.849016 6181 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.189:37283
I20260923 18:04:51.849344 6248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.189:37283 every 8 connection(s)
I20260923 18:04:51.850041 6181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/data/info.pb
I20260923 18:04:51.850354 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 6181
I20260923 18:04:51.850500 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-1/wal/instance
I20260923 18:04:51.850871 15918 external_mini_cluster.cc:1428] Setting key 4a93db5ff1c86aaca0479dfadcef24a1
I20260923 18:04:51.853302 6249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:51.858882 6249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.139.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.139.188@KRBTEST.COM" created.
Sep 23 18:04:51 dist-test-slave-vr57 krb5kdc[6093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186691, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.189@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:51.871049 6174 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } attempt: 3
I20260923 18:04:51.875730 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.861277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35362 (local address 127.15.139.190:34101)
0923 18:04:51.861428 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:51.861432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:51.862383 (+ 951us) server_negotiation.cc:407] Connection header received
0923 18:04:51.863493 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:51.863496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:51.863592 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:51.863658 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:51.865248 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.866108 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.867388 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.867805 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.867938 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:51.870877 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:51.870881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:51.870884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:51.870915 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:51.872591 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:51.873280 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:51.873285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:51.873287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:51.873386 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:51.873866 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:51.873869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:51.873870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:51.874081 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:51.874185 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:51.875426 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0923 18:04:51.875599 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 18:04:51.876353 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.861626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:34101 (local address 127.0.0.1:35362)
0923 18:04:51.862148 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.862256 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:51.863231 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:51.863856 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:51.863869 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:51.864406 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:51.865080 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:51.865096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.866274 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.866276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:51.867230 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:51.867259 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.867994 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.867997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:51.868190 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:51.868912 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:51.868939 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:51.870669 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:51.872706 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:51.872712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:51.872725 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:51.873151 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:51.873526 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:51.873533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:51.873536 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:51.873751 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:51.874220 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:51.874227 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:51.874492 (+ 265us) client_negotiation.cc:769] Sending connection context
0923 18:04:51.874674 (+ 182us) client_negotiation.cc:240] Negotiation successful
0923 18:04:51.875121 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":135,"threads_started":1}
I20260923 18:04:51.877844 6249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 18:04:51 dist-test-slave-vr57 krb5kdc[6093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186691, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.139.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.188.keytab.
Entry for principal kudu/127.15.139.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.188.keytab.
I20260923 18:04:51.883108 15918 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
/tmp/dist-test-taskwW7RpU/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.188:33745
--webserver_interface=127.15.139.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.188.keytab
--principal=kudu/127.15.139.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.139.148:35361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.139.190:34101,127.15.139.189:37283,127.15.139.188:33745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 18:04:51.883245 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.871602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:37283 (local address 127.0.0.1:34902)
0923 18:04:51.871782 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.871799 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:51.871887 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:51.873186 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:51.873192 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:51.873407 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:51.873649 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:51.873660 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.874930 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.874934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:51.875851 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:51.875863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.876710 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.876715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:51.876837 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:51.877367 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:51.877382 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:51.879041 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:51.881373 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:51.881375 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:51.881377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:51.881608 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:51.881999 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:51.882005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:51.882010 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:51.882111 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:51.882621 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:51.882624 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:51.882745 (+ 121us) client_negotiation.cc:769] Sending connection context
0923 18:04:51.882883 (+ 138us) client_negotiation.cc:240] Negotiation successful
0923 18:04:51.883042 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260923 18:04:51.883507 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.871797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34902 (local address 127.15.139.189:37283)
0923 18:04:51.872808 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:51.872817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:51.872837 (+ 20us) server_negotiation.cc:407] Connection header received
0923 18:04:51.872872 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:51.872876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:51.873045 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:51.873156 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:51.873780 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.874703 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.876001 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.876534 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.876647 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:51.879186 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:51.879193 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:51.879205 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:51.879245 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:51.881258 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:51.881773 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:51.881779 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:51.881782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:51.881877 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:51.882232 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:51.882238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:51.882242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:51.882503 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:51.882608 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:51.882959 (+ 351us) server_negotiation.cc:299] Negotiation successful
0923 18:04:51.883077 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":917,"thread_start_us":164,"threads_started":1}
I20260923 18:04:51.885543 6174 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 18:04:51 dist-test-slave-vr57 krb5kdc[6093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186691, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.189@KRBTEST.COM for kudu/127.15.139.189@KRBTEST.COM
I20260923 18:04:51.886339 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.886050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48312)
0923 18:04:51.886183 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.886245 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 18:04:51.887230 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.886890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48314)
0923 18:04:51.887077 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.887138 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260923 18:04:51.887530 6174 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:33745: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
I20260923 18:04:51.890446 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.878434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:37283 (local address 127.0.0.1:34912)
0923 18:04:51.878552 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.878564 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:51.878654 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:51.879274 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:51.879280 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:51.879491 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:51.879655 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:51.879662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.880782 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.880785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:51.881544 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:51.881551 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.882245 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.882248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:51.884202 (+ 1954us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:51.884772 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:51.884794 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:51.886373 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:51.888762 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:51.888769 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:51.888771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:51.888980 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:51.889314 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:51.889317 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:51.889319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:51.889406 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:51.889834 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:51.889837 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:51.889937 (+ 100us) client_negotiation.cc:769] Sending connection context
0923 18:04:51.890071 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 18:04:51.890244 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":1776}
I20260923 18:04:51.890466 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.878709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34912 (local address 127.15.139.189:37283)
0923 18:04:51.879035 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:51.879045 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:51.879060 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:04:51.879098 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:51.879104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:51.879160 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:51.879253 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:51.879803 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.880669 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.881736 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:51.882107 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:51.882248 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:51.886556 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:51.886561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:51.886564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:51.886593 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:51.888633 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:51.889102 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:51.889106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:51.889108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:51.889198 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:51.889509 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:51.889512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:51.889514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:51.889721 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:51.889844 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:51.890122 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 18:04:51.890309 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":252,"thread_start_us":122,"threads_started":1}
I20260923 18:04:51.891347 6249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:04:51.892231 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.891943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48322)
0923 18:04:51.892078 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.892166 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260923 18:04:51.892467 6197 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
I20260923 18:04:51.893488 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.893255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48324)
0923 18:04:51.893377 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.893427 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260923 18:04:51.893667 6249 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:33745: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
I20260923 18:04:51.918727 6174 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } attempt: 1
I20260923 18:04:51.919737 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.919357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48334)
0923 18:04:51.919563 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.919639 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260923 18:04:51.920612 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.920371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48350)
0923 18:04:51.920488 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.920542 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260923 18:04:51.920847 6174 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:33745: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
I20260923 18:04:51.942952 6249 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } attempt: 1
I20260923 18:04:51.944063 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.943718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48352)
0923 18:04:51.943891 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.943966 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260923 18:04:51.944844 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.944622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48358)
0923 18:04:51.944729 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.944783 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260923 18:04:51.945041 6249 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:33745: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
I20260923 18:04:51.996009 6174 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } attempt: 2
I20260923 18:04:51.997292 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.996929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48360)
0923 18:04:51.997108 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.997192 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260923 18:04:51.998107 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:51.997851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48368)
0923 18:04:51.997979 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:51.998035 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260923 18:04:51.998317 6174 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:33745: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
W20260923 18:04:52.004570 6256 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 18:04:52.004864 6256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 18:04:52.004927 6256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 18:04:52.005000 6256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 18:04:52.009246 6256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 18:04:52.009320 6256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 18:04:52.009339 6256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 18:04:52.009357 6256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 18:04:52.009402 6256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 18:04:52.013239 6249 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } attempt: 2
I20260923 18:04:52.014186 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.013811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48378)
0923 18:04:52.013993 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:52.014072 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260923 18:04:52.014953 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.014737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48386)
0923 18:04:52.014842 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:52.014891 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260923 18:04:52.015115 6249 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.139.188:33745: Network error: Client connection negotiation failed: client connection to 127.15.139.188:33745: connect: Connection refused (error 111)
I20260923 18:04:52.015793 6256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.139.148:35361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.139.190:34101,127.15.139.189:37283,127.15.139.188:33745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/krb5kdc/kudu_127.15.139.188.keytab
--principal=kudu/127.15.139.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.139.188:33745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.139.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 23 Sep 2026 17:12:33 UTC on ccycloud.quasar-bzzyns.root.comops.site
build id 4667750
I20260923 18:04:52.016912 6256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 18:04:52.017961 6256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 18:04:52.026971 6265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:52.027038 6262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 18:04:52.026971 6263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 18:04:52.027201 6256 server_base.cc:1034] running on GCE node
I20260923 18:04:52.027930 6256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 18:04:52.029202 6256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 18:04:52.030426 6256 hybrid_clock.cc:648] HybridClock initialized: now 1790186692030392 us; error 52 us; skew 500 ppm
Sep 23 18:04:52 dist-test-slave-vr57 krb5kdc[6093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186692, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 18:04:52.033631 6256 init.cc:377] Logged in from keytab as kudu/127.15.139.188@KRBTEST.COM (short username kudu)
I20260923 18:04:52.034610 6256 webserver.cc:460] Webserver started at http://127.15.139.188:38039/ using document root <none> and password file <none>
I20260923 18:04:52.035280 6256 fs_manager.cc:359] Metadata directory not provided
I20260923 18:04:52.035390 6256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 18:04:52.035702 6256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 18:04:52.037954 6256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/data/instance:
uuid: "31fa063bcb4f4c68aaf5f08e433c6328"
format_stamp: "Formatted at 2026-09-23 18:04:52 on dist-test-slave-vr57"
server_key: "3460e6c7be2cf6d19b52719792e02b69"
server_key_iv: "2a07f11c8b8a61d816e6fd274e51da0f"
server_key_version: "encryptionkey@0"
I20260923 18:04:52.038609 6256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/wal/instance:
uuid: "31fa063bcb4f4c68aaf5f08e433c6328"
format_stamp: "Formatted at 2026-09-23 18:04:52 on dist-test-slave-vr57"
server_key: "3460e6c7be2cf6d19b52719792e02b69"
server_key_iv: "2a07f11c8b8a61d816e6fd274e51da0f"
server_key_version: "encryptionkey@0"
I20260923 18:04:52.042956 6256 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 18:04:52.045943 6272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 18:04:52.047362 6256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 18:04:52.047557 6256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/data,/tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/wal
uuid: "31fa063bcb4f4c68aaf5f08e433c6328"
format_stamp: "Formatted at 2026-09-23 18:04:52 on dist-test-slave-vr57"
server_key: "3460e6c7be2cf6d19b52719792e02b69"
server_key_iv: "2a07f11c8b8a61d816e6fd274e51da0f"
server_key_version: "encryptionkey@0"
I20260923 18:04:52.047710 6256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 18:04:52.074954 6256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 18:04:52.079044 6256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 18:04:52.079351 6256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 18:04:52.087447 6256 rpc_server.cc:307] RPC server started. Bound to: 127.15.139.188:33745
I20260923 18:04:52.087837 6325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.139.188:33745 every 8 connection(s)
I20260923 18:04:52.088698 6256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/data/info.pb
I20260923 18:04:52.091707 6326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 18:04:52.091832 15918 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu as pid 6256
I20260923 18:04:52.091962 15918 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwW7RpU/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790186398863621-15918-0/minicluster-data/master-2/wal/instance
I20260923 18:04:52.092344 15918 external_mini_cluster.cc:1428] Setting key 1e4acced9406dcfbb1785bbdb8ca0143
I20260923 18:04:52.097579 6174 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } attempt: 3
I20260923 18:04:52.097369 6326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 18:04:52 dist-test-slave-vr57 krb5kdc[6093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186691, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.190@KRBTEST.COM for kudu/127.15.139.188@KRBTEST.COM
Sep 23 18:04:52 dist-test-slave-vr57 krb5kdc[6093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.188@KRBTEST.COM
I20260923 18:04:52.106329 6249 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } attempt: 3
Sep 23 18:04:52 dist-test-slave-vr57 krb5kdc[6093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186692, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.188@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:52.111972 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.098084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48398)
0923 18:04:52.098220 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:52.098238 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:52.098325 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:52.098805 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:52.098808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:52.099012 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:52.099150 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:52.099157 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.100357 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.100361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:52.101124 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:52.101135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.101827 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.101832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:52.101935 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.102619 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:52.102639 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:52.104344 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:52.107309 (+ 2965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:52.107318 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:52.107324 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:52.108155 (+ 831us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:52.110300 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:52.110305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:52.110311 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:52.110425 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:52.111332 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:52.111335 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:52.111441 (+ 106us) client_negotiation.cc:769] Sending connection context
0923 18:04:52.111615 (+ 174us) client_negotiation.cc:240] Negotiation successful
0923 18:04:52.111775 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":108}
I20260923 18:04:52.112377 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.099392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35370 (local address 127.15.139.190:34101)
0923 18:04:52.099552 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.099557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.099939 (+ 382us) server_negotiation.cc:407] Connection header received
0923 18:04:52.100197 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.100200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.100252 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.100355 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.101241 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.102002 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.103676 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.103923 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.104058 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.107568 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:52.107573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:52.107576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:52.107611 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:52.109896 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.110561 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.110565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.110567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.110648 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.110940 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.110944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.110946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.111256 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:52.111360 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:52.112104 (+ 744us) server_negotiation.cc:299] Negotiation successful
0923 18:04:52.112235 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":55}
Sep 23 18:04:52 dist-test-slave-vr57 krb5kdc[6093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186691, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.189@KRBTEST.COM for kudu/127.15.139.188@KRBTEST.COM
I20260923 18:04:52.114135 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.099351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.190:34101 (local address 127.0.0.1:35370)
0923 18:04:52.099798 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:52.099855 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:52.100027 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:52.100462 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:52.100468 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:52.100955 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:52.101096 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:52.101105 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.102713 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.102717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:52.103480 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:52.103491 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.104227 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.104230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:52.104316 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.104981 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:52.105001 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:52.107314 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:52.110046 (+ 2732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:52.110054 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:52.110062 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:52.110433 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:52.110762 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:52.110765 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:52.110766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:52.110855 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:52.111633 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:52.111640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:52.111810 (+ 170us) client_negotiation.cc:769] Sending connection context
0923 18:04:52.112554 (+ 744us) client_negotiation.cc:240] Negotiation successful
0923 18:04:52.112965 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"spinlock_wait_cycles":4480,"thread_start_us":97,"threads_started":1}
I20260923 18:04:52.114135 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.098138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.15.139.188:33745)
0923 18:04:52.098535 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.098546 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.098563 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:04:52.098601 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.098604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.098647 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.098755 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.099299 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.100235 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.101257 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.101687 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.102033 (+ 346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.104519 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:52.104529 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:52.104548 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:52.104588 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:52.107092 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.108962 (+ 1870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.108973 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.108983 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.110166 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.110576 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.110579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.110580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.111209 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:52.111366 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:52.112413 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0923 18:04:52.112850 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":329,"spinlock_wait_cycles":4480,"thread_start_us":178,"threads_started":1}
I20260923 18:04:52.114135 6329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.093942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48394 (local address 127.15.139.188:33745)
0923 18:04:52.094419 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.094438 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.094478 (+ 40us) server_negotiation.cc:407] Connection header received
0923 18:04:52.095333 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.095352 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.095854 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.096186 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.097302 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.099357 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.100752 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.101516 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.101741 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.105312 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:52.105319 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:52.105323 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:52.105355 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:52.109342 (+ 3987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.110080 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.110084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.110086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.110230 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.110654 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.110657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.110659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.111545 (+ 886us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:52.113031 (+ 1486us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:52.113111 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 18:04:52.113248 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":504,"server-negotiator.queue_time_us":322,"thread_start_us":131,"threads_started":1}
I20260923 18:04:52.115710 6326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:04:52.117997 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.106881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48400)
0923 18:04:52.107057 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:52.107079 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:52.107189 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:52.107864 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:52.107867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:52.108156 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:52.108316 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:52.108327 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.109445 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.109451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:52.110302 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:52.110314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.110957 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.110961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:52.111050 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.111653 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:52.111674 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:52.113234 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:52.115708 (+ 2474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:52.115712 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:52.115715 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:52.116017 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:52.116549 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:52.116555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:52.116560 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:52.116679 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:52.117416 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:52.117419 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:52.117557 (+ 138us) client_negotiation.cc:769] Sending connection context
0923 18:04:52.117663 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 18:04:52.117825 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260923 18:04:52.118979 6332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.107145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48400 (local address 127.15.139.188:33745)
0923 18:04:52.107576 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.107584 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.107596 (+ 12us) server_negotiation.cc:407] Connection header received
0923 18:04:52.107633 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.107634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.107709 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.108443 (+ 734us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.108542 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.109299 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.110447 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.110812 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.113377 (+ 2565us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.113485 (+ 108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:52.113488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:52.113491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:52.113507 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:52.115514 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.116311 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.116315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.116317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.116419 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.117015 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.117020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.117031 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.117290 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:52.118405 (+ 1115us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:52.118505 (+ 100us) server_negotiation.cc:299] Negotiation successful
0923 18:04:52.118723 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":329,"thread_start_us":192,"threads_started":1}
I20260923 18:04:52.119835 6174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3: Bootstrap starting.
I20260923 18:04:52.122687 6249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c: Bootstrap starting.
I20260923 18:04:52.123274 6174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3: Neither blocks nor log segments found. Creating new log.
Sep 23 18:04:52 dist-test-slave-vr57 krb5kdc[6093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186692, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.188@KRBTEST.COM for kudu/127.15.139.189@KRBTEST.COM
Sep 23 18:04:52 dist-test-slave-vr57 krb5kdc[6093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.190@KRBTEST.COM
I20260923 18:04:52.124537 6174 log.cc:824] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:52.126135 6249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:52.126996 6249 log.cc:824] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:52.129267 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.116963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34926 (local address 127.15.139.189:37283)
0923 18:04:52.117116 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.117120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.117385 (+ 265us) server_negotiation.cc:407] Connection header received
0923 18:04:52.117593 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.117599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.117652 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.117703 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.118911 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.119723 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.121052 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.121410 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.121507 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.125061 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:52.125065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:52.125068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:52.125094 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:52.127185 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.127839 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.127843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.127845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.127931 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.128302 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.128305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.128307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.128542 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:52.128615 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:52.128979 (+ 364us) server_negotiation.cc:299] Negotiation successful
0923 18:04:52.129120 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 18:04:52.129525 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.117912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35384 (local address 127.15.139.190:34101)
0923 18:04:52.118027 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.118031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.118196 (+ 165us) server_negotiation.cc:407] Connection header received
0923 18:04:52.118267 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.118269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.118323 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.118447 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.119291 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.119975 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.121078 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.121410 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.121507 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.125022 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:52.125026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:52.125029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:52.125058 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:52.127017 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.127715 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.127718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.127720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.127817 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.128191 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.128197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.128201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.128424 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:52.128593 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:52.129241 (+ 648us) server_negotiation.cc:299] Negotiation successful
0923 18:04:52.129360 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 18:04:52.129602 6174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3: No bootstrap required, opened a new log
I20260923 18:04:52.129607 6249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c: No bootstrap required, opened a new log
I20260923 18:04:52.130564 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.116764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.189:37283 (local address 127.0.0.1:34926)
0923 18:04:52.117311 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:52.117327 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:52.117463 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:52.117831 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:52.117834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:52.118640 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:52.118777 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:52.118784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.120018 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.120024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:52.120903 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:52.120914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.121703 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.121707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:52.121806 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.122449 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:52.122466 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:52.124900 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:52.127346 (+ 2446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:52.127349 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:52.127352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:52.127711 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:52.128084 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:52.128087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:52.128089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:52.128192 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:52.128689 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:52.128692 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:52.128788 (+ 96us) client_negotiation.cc:769] Sending connection context
0923 18:04:52.129573 (+ 785us) client_negotiation.cc:240] Negotiation successful
0923 18:04:52.130007 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":142,"threads_started":1}
I20260923 18:04:52.131740 6326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } has no permanent_uuid. Determining permanent_uuid...
I20260923 18:04:52.134292 6249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } }
I20260923 18:04:52.134357 6174 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } }
I20260923 18:04:52.134558 6249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003312950eea4fedab80fc91ee9a3d3c, State: Initialized, Role: FOLLOWER
I20260923 18:04:52.134611 6174 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31542cf98e9544768c0936b332f15be3, State: Initialized, Role: FOLLOWER
I20260923 18:04:52.134945 6249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } }
I20260923 18:04:52.134960 6174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } }
I20260923 18:04:52.136724 6174 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:52.137274 6249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [sys.catalog]: configured and running, proceeding with master startup.
Sep 23 18:04:52 dist-test-slave-vr57 krb5kdc[6093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.139.189@KRBTEST.COM
Sep 23 18:04:52 dist-test-slave-vr57 krb5kdc[6093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790186692, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.139.188@KRBTEST.COM for kudu/127.15.139.188@KRBTEST.COM
I20260923 18:04:52.142107 6337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } } }
I20260923 18:04:52.142234 6337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:04:52.142439 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.132223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.139.188:33745 (local address 127.0.0.1:48414)
0923 18:04:52.132586 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0923 18:04:52.132599 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 18:04:52.132705 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 18:04:52.132995 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 18:04:52.132997 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 18:04:52.133229 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0923 18:04:52.133323 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:52.133331 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.134299 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.134302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:52.135045 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 18:04:52.135056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.135849 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.135857 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 18:04:52.135938 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.136368 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 18:04:52.136381 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 18:04:52.138602 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 18:04:52.140844 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:52.140847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:52.140849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:52.141105 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:52.141393 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 18:04:52.141396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 18:04:52.141397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 18:04:52.141480 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 18:04:52.141926 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 18:04:52.141929 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 18:04:52.142012 (+ 83us) client_negotiation.cc:769] Sending connection context
0923 18:04:52.142119 (+ 107us) client_negotiation.cc:240] Negotiation successful
0923 18:04:52.142273 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307}
I20260923 18:04:52.143641 6332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.132417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48414 (local address 127.15.139.188:33745)
0923 18:04:52.132514 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.132518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.132643 (+ 125us) server_negotiation.cc:407] Connection header received
0923 18:04:52.132791 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.132794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.132832 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.132878 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.133437 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.134180 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.135178 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.135562 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.135664 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.138769 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:52.138773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:52.138776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:52.138807 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:52.140722 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.141211 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.141214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.141215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.141300 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.141569 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.141572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.141574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.141799 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:52.141893 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:52.143345 (+ 1452us) server_negotiation.cc:299] Negotiation successful
0923 18:04:52.143478 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 18:04:52.144613 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.130999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34934 (local address 127.15.139.189:37283)
0923 18:04:52.131172 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.131176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.131191 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:04:52.131224 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.131227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.131268 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.131350 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.132217 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.132995 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.134162 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.134477 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.134557 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.139989 (+ 5432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:52.140000 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:52.140007 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:52.140041 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:52.141793 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.143042 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.143045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.143048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.143153 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.143519 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.143522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.143524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.143734 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:52.143919 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:52.144317 (+ 398us) server_negotiation.cc:299] Negotiation successful
0923 18:04:52.144447 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260923 18:04:52.146804 6326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328: Bootstrap starting.
W20260923 18:04:52.147568 6353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:04:52.147647 6353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:04:52.145560 6106 master_runner.cc:186] Error getting master registration for 127.15.139.188:33745: OK, instance_id { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" instance_seqno: 1790186692085282 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 18:04:52.149030 6106 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 18:04:52.149398 6326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328: Neither blocks nor log segments found. Creating new log.
I20260923 18:04:52.150354 6326 log.cc:824] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328: Log is configured to *not* fsync() on all Append() calls
I20260923 18:04:52.150967 6336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } } }
I20260923 18:04:52.151114 6336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:04:52.151906 15918 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 18:04:52.153147 6148 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "31542cf98e9544768c0936b332f15be3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35384
I20260923 18:04:52.153376 6148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 18:04:52.153482 6148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [term 0 FOLLOWER]: Advancing to term 1
W20260923 18:04:52.153949 6359 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:04:52.154013 6359 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:04:52.154033 6326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328: No bootstrap required, opened a new log
I20260923 18:04:52.155009 6148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } }
I20260923 18:04:52.151976 6181 master_runner.cc:186] Error getting master registration for 127.15.139.188:33745: OK, instance_id { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" instance_seqno: 1790186692085282 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 18:04:52.155527 6181 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 18:04:52.156111 6148 leader_election.cc:288] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [CANDIDATE]: Term 1 election: Requested vote from peers 003312950eea4fedab80fc91ee9a3d3c (127.15.139.189:37283), 31fa063bcb4f4c68aaf5f08e433c6328 (127.15.139.188:33745)
I20260923 18:04:52.156608 6223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31542cf98e9544768c0936b332f15be3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "003312950eea4fedab80fc91ee9a3d3c"
I20260923 18:04:52.156847 6223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:52.158380 6223 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31542cf98e9544768c0936b332f15be3 in term 1.
I20260923 18:04:52.158605 6326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } }
I20260923 18:04:52.158883 6124 leader_election.cc:302] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003312950eea4fedab80fc91ee9a3d3c, 31542cf98e9544768c0936b332f15be3; no voters:
I20260923 18:04:52.158941 6326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31fa063bcb4f4c68aaf5f08e433c6328, State: Initialized, Role: FOLLOWER
I20260923 18:04:52.159184 6337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 18:04:52.159319 6326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } }
I20260923 18:04:52.159446 6337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [term 1 LEADER]: Becoming Leader. State: Replica: 31542cf98e9544768c0936b332f15be3, State: Running, Role: LEADER
I20260923 18:04:52.159844 6337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } }
I20260923 18:04:52.160861 6326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 18:04:52.160940 6300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31542cf98e9544768c0936b332f15be3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "31fa063bcb4f4c68aaf5f08e433c6328"
I20260923 18:04:52.161187 6300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [term 0 FOLLOWER]: Advancing to term 1
I20260923 18:04:52.162271 6367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31542cf98e9544768c0936b332f15be3. Latest consensus state: current_term: 1 leader_uuid: "31542cf98e9544768c0936b332f15be3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } } }
I20260923 18:04:52.162511 6367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:52.162705 6300 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31542cf98e9544768c0936b332f15be3 in term 1.
I20260923 18:04:52.172741 6371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 18:04:52.173005 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.162156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34940 (local address 127.15.139.189:37283)
0923 18:04:52.162309 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.162313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.164127 (+ 1814us) server_negotiation.cc:407] Connection header received
0923 18:04:52.164314 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.164317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.164377 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.164457 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.165395 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.166148 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.167026 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.167392 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.167534 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.169088 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:52.169092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:52.169095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:52.169120 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:52.170860 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.171353 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.171357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.171359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.171449 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.171753 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.171757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.171758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.172012 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:52.172139 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:52.172701 (+ 562us) server_negotiation.cc:299] Negotiation successful
0923 18:04:52.172828 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260923 18:04:52.174340 6371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 18:04:52.176716 6368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } } }
I20260923 18:04:52.176862 6368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 18:04:52.180316 6384 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 18:04:52.180404 6384 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 18:04:52.183056 6332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.163694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48430 (local address 127.15.139.188:33745)
0923 18:04:52.163821 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.163828 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.172208 (+ 8380us) server_negotiation.cc:407] Connection header received
0923 18:04:52.172254 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.172257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.172302 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.172379 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.173975 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.174643 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.175590 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.175970 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.176064 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.179243 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:52.179250 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:52.179255 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:52.179319 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:52.181173 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.181680 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.181683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.181685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.181773 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.182151 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.182153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.182154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.182337 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:52.182511 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:52.182746 (+ 235us) server_negotiation.cc:299] Negotiation successful
0923 18:04:52.182866 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 18:04:52.184787 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:52.163139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35386 (local address 127.15.139.190:34101)
0923 18:04:52.163566 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.163571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.171863 (+ 8292us) server_negotiation.cc:407] Connection header received
0923 18:04:52.171976 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.171979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.172032 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.172116 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.176367 (+ 4251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.177133 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.178053 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:52.178403 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:52.178549 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:52.181052 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 18:04:52.181056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 18:04:52.181059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 18:04:52.181101 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 18:04:52.182790 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.183378 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.183382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.183384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.183476 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 18:04:52.183834 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 18:04:52.183837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 18:04:52.183839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 18:04:52.184074 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 18:04:52.184178 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:52.184475 (+ 297us) server_negotiation.cc:299] Negotiation successful
0923 18:04:52.184594 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
I20260923 18:04:52.191540 6223 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [term 1 FOLLOWER]: Refusing update from remote peer 31542cf98e9544768c0936b332f15be3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:52.192052 6300 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [term 1 FOLLOWER]: Refusing update from remote peer 31542cf98e9544768c0936b332f15be3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 18:04:52.192255 6367 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:52.192840 6367 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 18:04:52.196314 6336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31542cf98e9544768c0936b332f15be3. Latest consensus state: current_term: 1 leader_uuid: "31542cf98e9544768c0936b332f15be3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } } }
I20260923 18:04:52.196457 6336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:04:52.196741 6368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31542cf98e9544768c0936b332f15be3. Latest consensus state: current_term: 1 leader_uuid: "31542cf98e9544768c0936b332f15be3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } } }
I20260923 18:04:52.196895 6368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:04:52.199713 6337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31542cf98e9544768c0936b332f15be3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } } }
I20260923 18:04:52.199867 6337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:52.200176 6385 mvcc.cc:205] Tried to move back new op lower bound from 7332604691207979008 to 7332604691090448384. Current Snapshot: MvccSnapshot[applied={T|T < 7332604691207979008}]
I20260923 18:04:52.200451 6368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31542cf98e9544768c0936b332f15be3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } } }
I20260923 18:04:52.200585 6368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:04:52.202025 6371 catalog_manager.cc:1347] Generated new cluster ID: 5b061a90941f4e67a0df84295d547e40
I20260923 18:04:52.202101 6371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 18:04:52.210510 6388 mvcc.cc:205] Tried to move back new op lower bound from 7332604691207979008 to 7332604691090448384. Current Snapshot: MvccSnapshot[applied={T|T < 7332604691207979008}]
I20260923 18:04:52.215137 6337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31542cf98e9544768c0936b332f15be3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } } }
I20260923 18:04:52.215289 6337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3 [sys.catalog]: This master's current role is: LEADER
I20260923 18:04:52.215987 6371 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 18:04:52.216804 6336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31542cf98e9544768c0936b332f15be3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31542cf98e9544768c0936b332f15be3" member_type: VOTER last_known_addr { host: "127.15.139.190" port: 34101 } } peers { permanent_uuid: "003312950eea4fedab80fc91ee9a3d3c" member_type: VOTER last_known_addr { host: "127.15.139.189" port: 37283 } } peers { permanent_uuid: "31fa063bcb4f4c68aaf5f08e433c6328" member_type: VOTER last_known_addr { host: "127.15.139.188" port: 33745 } } }
I20260923 18:04:52.216959 6336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c [sys.catalog]: This master's current role is: FOLLOWER
I20260923 18:04:52.217404 6371 catalog_manager.cc:1504] Loading token signing keys...
I20260923 18:04:52.218048 6387 mvcc.cc:205] Tried to move back new op lower bound from 7332604691207979008 to 7332604691090448384. Current Snapshot: MvccSnapshot[applied={T|T < 7332604691207979008}]
I20260923 18:04:52.228557 6371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31542cf98e9544768c0936b332f15be3: Generated new TSK 0
I20260923 18:04:52.229413 6371 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 18:04:52.262809 6332 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.259372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48444 (local address 127.15.139.188:33745)
0923 18:04:52.259534 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.259538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.261655 (+ 2117us) server_negotiation.cc:407] Connection header received
0923 18:04:52.261873 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.261877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.261932 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.261986 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.262650 (+ 664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48444: BlockingRecv error: recv got EOF from 127.0.0.1:48444 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260923 18:04:52.264531 6177 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.259909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35392 (local address 127.15.139.190:34101)
0923 18:04:52.260035 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.260039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.263487 (+ 3448us) server_negotiation.cc:407] Connection header received
0923 18:04:52.263723 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.263729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.263787 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.263841 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.264398 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35392: BlockingRecv error: recv got EOF from 127.0.0.1:35392 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260923 18:04:52.266021 6255 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.260373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34954 (local address 127.15.139.189:37283)
0923 18:04:52.260492 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.260496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.265159 (+ 4663us) server_negotiation.cc:407] Connection header received
0923 18:04:52.265324 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.265327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.265377 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.265426 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.265901 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34954: BlockingRecv error: recv got EOF from 127.0.0.1:34954 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260923 18:04:52.270716 6255 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.269183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34968 (local address 127.15.139.189:37283)
0923 18:04:52.269283 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.269287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.269754 (+ 467us) server_negotiation.cc:407] Connection header received
0923 18:04:52.269971 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.269974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.270049 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.270103 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.270618 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34968: BlockingRecv error: recv got EOF from 127.0.0.1:34968 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260923 18:04:52.276544 6255 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.275082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34980 (local address 127.15.139.189:37283)
0923 18:04:52.275185 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.275189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.275715 (+ 526us) server_negotiation.cc:407] Connection header received
0923 18:04:52.275888 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.275891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.275937 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.275986 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.276448 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34980: BlockingRecv error: recv got EOF from 127.0.0.1:34980 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260923 18:04:52.282368 6255 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.280924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34990 (local address 127.15.139.189:37283)
0923 18:04:52.281021 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.281025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.281520 (+ 495us) server_negotiation.cc:407] Connection header received
0923 18:04:52.281714 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.281717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.281763 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.281810 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.282271 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34990: BlockingRecv error: recv got EOF from 127.0.0.1:34990 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260923 18:04:52.290458 6255 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.288738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35000 (local address 127.15.139.189:37283)
0923 18:04:52.288885 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.288889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.289474 (+ 585us) server_negotiation.cc:407] Connection header received
0923 18:04:52.289685 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.289688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.289742 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.289793 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.290338 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35000: BlockingRecv error: recv got EOF from 127.0.0.1:35000 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260923 18:04:52.303082 6255 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.301135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35016 (local address 127.15.139.189:37283)
0923 18:04:52.301310 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.301315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.301978 (+ 663us) server_negotiation.cc:407] Connection header received
0923 18:04:52.302186 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.302190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.302245 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.302316 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.302938 (+ 622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35016: BlockingRecv error: recv got EOF from 127.0.0.1:35016 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260923 18:04:52.324618 6255 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.322206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35018 (local address 127.15.139.189:37283)
0923 18:04:52.322383 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.322391 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.323277 (+ 886us) server_negotiation.cc:407] Connection header received
0923 18:04:52.323641 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.323645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.323726 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.323792 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.324472 (+ 680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35018: BlockingRecv error: recv got EOF from 127.0.0.1:35018 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260923 18:04:52.361753 6255 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.359675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.15.139.189:37283)
0923 18:04:52.359831 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.359836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.360521 (+ 685us) server_negotiation.cc:407] Connection header received
0923 18:04:52.360750 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.360753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.360814 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.360868 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.361606 (+ 738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35024: BlockingRecv error: recv got EOF from 127.0.0.1:35024 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260923 18:04:52.429842 6255 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.428753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.15.139.189:37283)
0923 18:04:52.428927 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.428930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.429031 (+ 101us) server_negotiation.cc:407] Connection header received
0923 18:04:52.429139 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.429141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.429192 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.429280 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.429742 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35040: BlockingRecv error: recv got EOF from 127.0.0.1:35040 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260923 18:04:52.561223 6255 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.560047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35054 (local address 127.15.139.189:37283)
0923 18:04:52.560218 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.560222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.560255 (+ 33us) server_negotiation.cc:407] Connection header received
0923 18:04:52.560437 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.560440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.560491 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.560581 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.561092 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35054: BlockingRecv error: recv got EOF from 127.0.0.1:35054 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260923 18:04:52.820858 6255 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 18:04:52.819567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35068 (local address 127.15.139.189:37283)
0923 18:04:52.819726 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:52.819731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:52.819855 (+ 124us) server_negotiation.cc:407] Connection header received
0923 18:04:52.820064 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:52.820072 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:52.820123 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:52.820220 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 18:04:52.820715 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35068: BlockingRecv error: recv got EOF from 127.0.0.1:35068 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 18:04:53.155375 6359 catalog_manager.cc:1259] Loaded cluster ID: 5b061a90941f4e67a0df84295d547e40
I20260923 18:04:53.155471 6359 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c: loading cluster ID for follower catalog manager: success
I20260923 18:04:53.157217 6359 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c: acquiring CA information for follower catalog manager: success
I20260923 18:04:53.158340 6359 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 003312950eea4fedab80fc91ee9a3d3c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 18:04:53.181706 6384 catalog_manager.cc:1259] Loaded cluster ID: 5b061a90941f4e67a0df84295d547e40
I20260923 18:04:53.181792 6384 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328: loading cluster ID for follower catalog manager: success
I20260923 18:04:53.183421 6384 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328: acquiring CA information for follower catalog manager: success
I20260923 18:04:53.184553 6384 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 31fa063bcb4f4c68aaf5f08e433c6328: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 18:04:53.340051 6461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:53.335293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35076 (local address 127.15.139.189:37283)
0923 18:04:53.335677 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:53.335681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:53.335698 (+ 17us) server_negotiation.cc:407] Connection header received
0923 18:04:53.335823 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:53.335825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:53.335951 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:53.336056 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:53.336360 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:53.336948 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:53.338110 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:53.338429 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:53.338560 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:53.339036 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:53.339579 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:53.339724 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:53.339766 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 18:04:53.339853 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":77,"threads_started":1}
I20260923 18:04:53.347137 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:53.343199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48448 (local address 127.15.139.188:33745)
0923 18:04:53.343451 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:53.343455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:53.343468 (+ 13us) server_negotiation.cc:407] Connection header received
0923 18:04:53.343603 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:53.343610 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:53.343731 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:53.343845 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:53.344104 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:53.344630 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:53.345524 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:53.345781 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:53.345922 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:53.346243 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:53.346690 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:53.346837 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:53.346899 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 18:04:53.346959 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":113,"threads_started":1}
I20260923 18:04:53.354727 6461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:53.350879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35092 (local address 127.15.139.189:37283)
0923 18:04:53.351003 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:53.351008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:53.351023 (+ 15us) server_negotiation.cc:407] Connection header received
0923 18:04:53.351075 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:53.351078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:53.351127 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:53.351235 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:53.351574 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:53.352301 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:53.353470 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:53.353833 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:53.353942 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:53.354205 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:53.354283 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:53.354363 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:53.354491 (+ 128us) server_negotiation.cc:299] Negotiation successful
0923 18:04:53.354545 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 18:04:53.355049 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:53.350721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48458 (local address 127.15.139.188:33745)
0923 18:04:53.350836 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:53.350840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:53.351262 (+ 422us) server_negotiation.cc:407] Connection header received
0923 18:04:53.351432 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:53.351435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:53.351484 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:53.351581 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:53.352039 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:53.352688 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:53.353764 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:53.354098 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:53.354200 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:53.354547 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:53.354636 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:53.354684 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:53.354852 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 18:04:53.354913 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 18:04:53.356683 6472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 18:04:53.350476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35398 (local address 127.15.139.190:34101)
0923 18:04:53.350756 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0923 18:04:53.350760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 18:04:53.350874 (+ 114us) server_negotiation.cc:407] Connection header received
0923 18:04:53.351274 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 18:04:53.351277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 18:04:53.351460 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 18:04:53.351610 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 18:04:53.352810 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:53.353537 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:53.354669 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 18:04:53.355047 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 18:04:53.355131 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 18:04:53.356033 (+ 902us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 18:04:53.356328 (+ 295us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 18:04:53.356406 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 18:04:53.356512 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 18:04:53.356565 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":89,"threads_started":1}
I20260923 18:04:53.359869 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 6106
I20260923 18:04:53.366570 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 6181
I20260923 18:04:53.372727 15918 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwW7RpU/build/debug/bin/kudu with pid 6256
2026-09-23T18:04:53Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3105 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6269 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (294501 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST